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ABSTRACT 

Technology education is a field of study which seeks to promote technological literacy 
for all students. Some recent research in the field has focused on integrating content and 
methodology from engineering design into technology education classes, particularly at 
the secondary level. This study contributes to the research base in technology education on 
the subject of incorporating the engineering design process into the technology education 
curriculum. It addressed the need for the development of a framework for understanding 
engineering design and the related academic concepts that can be used by professionals in 
the field of technology education seeking to incorporate the engineering design process into 
the technology education curriculum. The purpose of this study was to address the question 
“What are the essential aspects and related academic concepts of an engineering design process 
in secondary technology education curriculum for the purpose of establishing technological 
literacy?” 

A four-round Delphi process was the research methodology employed in this study to 
give multiple opportunities for the group opinion to coalesce. The resulting data from the Delphi 
process was analyzed and categorized. Only those items that met strictest criteria for high median 
score, low interquartile range, and consensus were accepted as very important and considered in 
the conclusions and recommendations. Participants in this study identified forty-eight items that 
met these strict requirements. 


The conclusions made from this study were related to the integration of engineering design 
into secondary technology education classes. The recommendations fell into three categories: 
future research, instructional delivery methods, and teacher preparation. 

INDEX WORDS: technology education, engineering design process, secondary education, 

mental processes, and Delphi process 


ESSENTIAL ASPECTS AND RELATED ACADEMIC CONCEPTS OP AN ENGINEERING 


DESIGN CURRICULUM IN SECONDARY TECHNOLOGY EDUCATION 


by 


PHILLIP CAMERON SMITH, JR. 

B.S. University of Georgia 1995 
M. Ed. University of Georgia 1998 


A Dissertation Submitted to the Graduate Paculty of The University of Georgia in Partial 
Pulfillment of the Requirements for the Degree 


DOCTOR OP EDUCATION 


ATHENS, GEORGIA 


2006 


© 2006 

Phillip Cameron Smith, Jr. 
All Rights Reserved 


ESSENTIAL ASPECTS AND RELATED ACADEMIC CONCEPTS OP AN ENGINEERING 


DESIGN CURRICULUM IN SECONDARY TECHNOLOGY EDUCATION 


by 


PHILLIP CAMERON SMITH, JR. 


Major Professor: Robert C. Wicklein 

Committee: Roger Hill 

David Gattie 
Sid Thompson 
Jay Rojewski 


Electronic Version Approved: 

Maureen Grasso 
Dean of the Graduate School 
The University of Georgia 
December 2006 


ACKNOWLEDGEMENTS 


This work is dedicated to my daughters, Emma and Alyssa. 

I would like to thank Dr. Wicklein and the rest of my committee members for their help in 
conceptualizing and completing this project. 

I am grateful to my parents, Phil and Einda Smith, for their prayers and tough-love upbringing. 

I would especially like to thank the love of my life, Jennifer, for her sacrificial help, tremendous 
understanding, and constant encouragement. This has truly been a team effort. 

Most importantly, I would like to thank God for the opportunities He has given me. He has given 
me an eternal relationship with Him through simple faith in Jesus, a wonderful family, and the 
chance to enjoy so many things in this life. 

Trust in the Lord with all your heart and 
lean not on your own understanding; 
in all your ways acknowledge Him, and 
He will make your paths straight. 

Proverbs 3:5-6 


IV 


TABLE OF CONTENTS 


Page 

ACKNOWEEDGEMENTS iv 

CHAPTER 

1 INTRODUCTION I 

Purpose of the Study 4 

Research Questions 4 

Conceptual Framework 5 

Significance of the Study 8 

Eimitations of the Study 10 

Summary II 

2 REVIEW OF EITERATURE 1 2 

Recent Technology Education Eiterature 15 

The Technology For All Americans Project 16 

Related Eiterature 24 

Engineering Design 26 

Accountability 31 

Mental Processes 32 

3 METHOD 35 

Resarch Design 35 

Population And Sample 38 

Round I 40 

Round 2 41 

Round 3 42 

Round 4 43 


V 


4 RESULTS 45 

Round 1 46 

Round 2 51 

Round 3 56 

Round 4 64 

Final Results 67 

Summary 72 

5 SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS FOR FUTURE 

RESEARCH 76 

General Summary 78 

Recommendations For Future Research 83 

REFERENCES 86 

APPENDICES 

A LETTER TO DR. CLIVE DYM 95 

B INITIAL CONTACT EMAIL 97 

C ROUND 1 SURVEY 99 

D ROUND 1 RESULTS - BY PARTICIPANT 10 1 

E ROUND 2 SURVEY 107 

F ROUND 3 SURVEY 1 1 2 

G ROUND 3 SURVEY INSTRUMENT 114 

H ROUND 4 EMAIL 122 

I ROUND 4 SURVEY INSTRUMENT 124 


VI 


CHAPTER 1 


INTRODUCTION 


Technology education is a field of study that seeks to promote technological literacy for 
all students. According to a recent study, in the United States technology education is part of the 
state framework for 38 states, there are approximately 35,909 middle or high school technology 
teachers, and technology education is most frequently an elective course (Meade & Dugger, 
2004). Indeed, “Technology education provides an opportunity for students to learn about the 
processes and knowledge related to technology that are needed to solve problems and extend 
human capabilities” (ITEA, 1996, p. 13). Wright and Luada (1993) defined technology education 
as a program designed to help students “develop an understanding and competence in designing, 
producing, and using technological products and systems, and in assessing the appropriateness of 
technological actions” (p. 4). 

The processes associated with technology have become key elements in technology 
education curriculum. A guiding influence in the development of this process-based curriculum 
has been the Technology for All Americans Project (Lewis, 1999; Loepp, 2004; Satchwell 
& Dugger, 1996; Wamsley 2003). With the publication of Technology for All Americans: A 
Rationale and Structure for the Study of Technology (ITEA, 1996), the suggested structure for 
the study of technology became the Universals of Technology which were identified as the 
processes, knowledge, and context associated with the development of technological systems: 

The processes are those actions that people undertake to create, invent, design, transform, 
produce, control, maintain, and use products or systems. The processes include the human 
activities of designing and developing technological systems; determining and controlling 
the behavior of technological systems; utilizing technological systems; and assessing the 
impacts and consequences of technological systems, (p. 16) 
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Thus, solving problems in the context of technological systems has been identified as a key 
aspect of the curriculum commonly associated with technology education (Sanders, 2001). 
Activities that involve solving problems have been called the “philosophical nucleus” (Dugger, 
1994, p.7) of technology education. Hill (1997) indicated that solving problems remains a major 
component of technological literacy. 

Although this structure has been provided for the field, various paradigms for delivering 
the curriculum of technology education exist (Bensen, 1995; Devore, 1968; Hatch, 1988; Maley, 
1973; Dyrenfurth, 1991; Savage & Sterry, 1990; Sydner & Hales, 1981; Wicklein & Rojewski, 
1999). The actual practice of technology education in the United States has been a somewhat 
eclectic mix of approaches and instructional methods (Foster & Wright, 1996; Sanders, 2001). 
Bensen (1995) found that some programs operated with a singular concept of technology 
in which all the supporting parts of the curriculum were related to the whole. Others were 
characterized by a plural concept in which various technologies are emphasized without an effort 
to relate them to the larger picture of technology and its effect in our world. The Standards for 
Technological Literacy (ITEA, 2000) do not mandate a particular curricular approach (Laporte, 
2001) and technology education programs in the United State employ various approaches (Boser, 
Palmer, & Daugherty, 1998; Satchwell & Dugger, 1996). This fragmented focus and lack of a 
clear curriculum framework have been detrimental to the potential of the field and have hindered 
efforts aimed at achieving the stated goals of technological literacy for all students. 

Emergence of Engineering Design in Technology Education 

In recent years there has been a growing emphasis in the literature of technology 
education not only on the process of problem solving but also, more recently, on the integration 
of subject matter from various disciplines within those activities (Cotton, 2002; Engstrom, 2001; 
ITEA, 2003; Merrill & Comerford, 2004). This development leads to many questions for the 
field of technology education regarding the nature of the curriculum being offered and the proper 
approaches to take in administering that curriculum in technology education classrooms. As the 
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field has begun to broaden its perspeetive and embrace ties with other disciplines, the topic of 
engineering design has begun to appear frequently in the literature (Bearing & Daugherty, 2004). 

Engineering design is not simply a frequent topic in the literature of technology 
education; it has already begun to be included in the curriculum in some areas. Some states have 
adopted technology education curriculum models that are pre-engineering in nature (Lewis, 
2004). Project Lead The Way, Career Academies that emphasize engineering, engineering 
magnet schools, and other conceptions such as the “Stony Brook” model are all examples of 
engineering content making its way into the middle and high school curricula (Lewis, 2004). 

Another recent development has been the funding and establishment by the National 
Science Loundation of the National Center for Engineering and Technology Education (NCETE). 
One of the main goals of this organization is to “work with engineering and technology educators 
to prepare them to introduce engineering design concepts in Grades 9-12” (Hailey, Erekson, 
Becker, & Thomas, 2005, p. 24). Currently, nine universities and numerous additional high 
schools across the country are NCETE Partners working to develop and disseminate materials, 
educate teachers, train future teacher educators, and facilitate relationships between the fields of 
engineering and technology education. The NCETE is also conducting several research studies 
beginning in 2005 related to engineering design in technology education. 

Conceptually, there are close ties between engineering and the field of public education 
known as technology education since “both engineering and technology treat solving practical 
problems as their philosophical nucleus” (Dugger, 1994, p. 7). In fact, engineering has been 
defined as “the profession in which knowledge of the mathematical and natural sciences 
gained by study, experience, and practice is applied with judgment to develop ways to utilize, 
economically, the materials and forces of nature for the benefit of mankind” (Accreditation 
Board Engineering & Technology, 1986, p. 1). Engineers have been described as “creative 
problem solvers, often imagining and designing new technologies as a means to solve problems” 
(Burghardt, 1999, p. 1). 
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Thus, solving problems is an intrinsic component of both technology education and the 
field of engineering. 

However, it is evident from an examination of the literature that there are certain aspects 
inherent to the engineering design process which are not included in technological problem 
solving (Pales, Kuetemeyer, & Brusic, 1999; Wright, 2002; Hailey et ah, 2005). Technology 
educators have indicated the need for further explanation of these differences (Wicklein & 
Gattie, 2004) in order to gain the expertise necessary to be able to incorporate the engineering 
design process in technology education classrooms. 

Purpose of the Study 

This study contributed to the research base in technology education on the subject of 
incorporating the engineering design process into the technology education curriculum. It 
addressed the need for the development of a framework for understanding engineering design 
and the related academic concepts that can be used by professionals in the field of technology 
education seeking to incorporate the engineering design process into the technology education 
curriculum. The purpose of this study was to address the question; What are the essential aspects 
and related academic concepts of an engineering design process in secondary technology 
education curriculum for the purpose of establishing technological literacy? 

Research Questions 

1 . What aspects of the engineering design process best equip secondary students 
to understand, manage, and solve technological problems? 

2. What mathematics concepts related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

3. What specific science principles related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

4. What specific skills, techniques, and engineering tools related to engineering 
design should secondary students use to understand, manage, and solve 
technological problems? 
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Conceptual Framework 

Many in the field of teehnology edueation are eurrently looking to engineering as 
a significant curriculum component (Lewis, 2004; Searcella, 2005). In fact, at the 2005 
International Teehnology Education Association conference in Kansas City, there were at 
least 23 presentations direetly related to engineering in teehnology education (ITEA, 2005). 
Many believe that ineluding engineering content will provide a method of incorporating cross- 
disciplinary, standards-based instruction while meeting the goal of technologieal literaey (Hailey 
et ah, 2005). The literature of teehnology education has begun to inelude numerous referenees 
supporting the inelusion of engineering design in technology education. Wicklein (2006) noted 
that a teehnology education curriculum that emphasizes engineering design is valuable beeause 

1 . Engineering Design is more understood and valued than teehnology education by the 
general populaee 

2. Engineering Design elevates the field of teehnology education to higher academie and 
teehnological levels 

3. Engineering Design provides a solid framework to design and organize eurriculum 

4. Engineering Design provides an ideal platform for integrating mathematics, science, 
and technology 

5. Engineering provides a foeused eurriculum which can lead to multiple career 
pathways for students. 

Even though there is support in the literature for ineluding engineering eontent in technology 
education, there are many questions left to answer. Technology teachers have indieated support 
for the inclusion of engineering eontent, but also a need for additional help in developing an 
engineering design currieulum and teaehing related eoneepts that have not typieally been a 
part of teehnology education (Wicklein & Gattie, 2004). Evidenee of the differenees between 
engineering design and teehnological problem solving can be seen elearly when the two 
proeesses are compared side by side. Table 1 contains the steps normally assoeiated with each 
process as found in the professional literature. 
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Table 1 


Comparison of Engineering Design and Technology Education Design Process 


Engineering Design Process 
(Eide, Jenison, Mashaw, & Northup, 2001) 

Identify the Need 
Define the Problem 
Search for Solutions 
Identify Constraints 
Specify Evaluation Criteria 
Generate Alternative Solutions 
Analysis 

Mathematical Predictions 
Optimization 
Decision 

Design Specifications 
Communication 


Technology Education Design Process 
(Hailey et AL, 2005) 

Defining a Problem 
Brainstorming 

Researching and Generating Ideas 
Identifying Criteria 
Specifying Constraints 
Exploring Possibilities 

Select an Approach and Develop a Design 
Proposal 

Building a Model or Prototype 
Testing & Evaluating the Design 
Refining the Design 
Communicating Results 


An obvious area of difference between the two design processes is that engineering design 
includes analysis, mathematical prediction, and optimization, identified as a major area of 
difference (Hailey et ah, 2005). Dugger (1994) indicated that technology is “guided by trial 
and error or skilled approaches derived from the concrete,” while engineering is “guided by a 
more analytical study with specific solutions recommended” (p. 7). Although there are many 
similarities between the two processes, there are no steps in the technology education design 
process that correspond to the analysis, mathematical prediction, and optimization components 
that are an integral part of engineering design. 

Mental Processes Involved in Problem Solving 
The importance of applying the skills of analyzing and predicting in the context of 
solving technological problems has been identified in the literature. Halfin (1973) defined 17 
mental processes technologists used in the process of solving problems. These processes were 
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re-examined and extended to form a list of 27 processes considered essential by a group made up 
primarily of engineers (Wicklein & Rojewski, 1999). A study involving factor analysis narrowed 
this somewhat unwieldy list down to a more accessible list of 5 areas of mental processes 
necessary for solving problems. A comparison of these five processes appears in Table 2. 


Table 2 

Mental Processes Identified by Factor Analysis 


researching 

managing 

creating 

computing 

monitoring data 

questions/ 

hypotheses 

measuring 

defining 

problem(s) 

predicting 

models/ 

prototypes 

values 

technology 

review 

innovating 

visualizing 

Observing 

establishing need 

communicating 

customer 

analysis 

modeling 



Need for Additional Research 

Technology teachers have indicated a need for additional help in gaining the requisite 
skills necessary in order to incorporate these aspects of engineering design. Table 3 includes 
partial results of a recent study (Wicklein & Gattie, 2004) on the topic of engineering design in 
technology education. 
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Table 3 


Engineering Design in Technology Education Survey (n=269) 


Mv instructional needs to teach enaineerina 
desian include: 

Agree 

Stronglv 

Agree 

Total Percentage 
Agree or Stronglv Agree 

Integrating the appropriate levels of 
mathematics and science into instructional 
content 

153 

101 

93.8% 

Gaining the appropriate levels of 
mathematics and science knowledge to 
teach engineering design 

136 

96 

86.3% 

Developing additional analytical (math) 
skills to be able to predict engineering 
results 

149 

84 

87.3% 


In problem-solving activities using the technological method, students are not typically required 
to “state explicitly the theory or equations needed to solve the problem” (Oakes, Leone, & 

Gunn, 2002, p. 201). Often the result is that “the related mathematics [are] explored only after 
the problems are solved” (Cotton, 2002, p. 29) if at all. Technology educators need to develop 
skills from related academic areas such as mathematics and science in order to incorporate ail the 
aspects of the engineering design process. 

Significance of the Study 

The results of this study will add to the growing literature in the field on the topic of 
engineering design and highlight the specific concepts from related areas (mathematics and 
science) that should be emphasized in secondary technology education. These concepts could 
then serve as a framework for creating an engineering design-focused curriculum that sustains 
the goals of the Standards for Technological Literacy (STL). One clear goal of the STL is 
integration of content from other areas in technology education classrooms for the purpose of 
understanding and managing technological systems, an ability which requires the use of concepts 
from science, mathematics, and other areas (ITEA, 2003). According to ITEA, “teachers 
[should] infuse technology programs with interdisciplinary linkages between technology and all 
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school subjects” (2003, p. 73). As engineering design is included in the curriculum, there will be 
many opportunities for “technology education to provide a role as the integrator of mathematics 
and science” (Hailey et ah, 2005, p. 25). This study will provide a greater understanding of 
engineering design so that a curriculum framework can be established for a technology education 
curriculum with an emphasis on engineering design. 

A curriculum emphasizing the mental processes technologists employ in solving 
problems has been suggested as a solid basis for creating a unifying curriculum framework for 
technology education (Wicklein & Rojewski, 1999): 

The mental processes are not developed as curriculum per se; however, they may serve as 
a basis for creating curriculum designs that may yield comprehensive and strategic means 
of employing critical thinking and problem-solving strategies for students. Curriculum 
that emphasizes technical content tends to be rather short lived and is constantly changing 
due to the rapid accumulation of knowledge and techniques used in business and industry. 
In comparison, the mental processes and techniques used in solving technological 
problems could remain rather consistent over time. (p. 40) 

This study will follow up and extend important research in the field on the mental processes 
involved in problem solving that are not typically part of the technological problem-solving 
process. This understanding could contribute to the creation of a framework that emphasizes 
the full range of mental processes involved in problem solving that have been identified in the 
literature and which require students to apply concepts from a range of cognitive areas. 

As the concepts identified in this study are developed into a framework and infused into 
an engineering design-focused curriculum, a rich contextual learning environment could result. 
Parnell (1995) identified such a contextual learning model: 

Learning for Acquisition of Knowledge. Students acquire information and retain it 
sufficiently to apply it toward or associate it with some real-life situation. 

Learning for Application. Students are actively engaged in practicing and processing 
what they learn within the context of varied real-life situations, performing authentic 
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tasks to gain an understanding of how the information applies in everyday life. 

Learning for Assimilation. Students demonstrate suffieient understanding of the content 
and context of what they are learning to apply knowledge and skills effectively to new 
situations. 

Learning for Association. The educational experience is organized around problems 
and themes rather than subject matter disciplines; students learn to transfer acquisition, 
application, and assimilation of knowledge to new problem-solving situations, (p. 15) 

A key component of contextual learning is providing a real-life situation in which the knowledge 
at hand is actually applied and students learn to associate the knowledge gained in a variety of 
subject areas to the problem at hand. A technology education curriculum with an engineering 
design focus could provide the context in which students experience the cross-disciplinary 
application of knowledge and skills to problems based on actual situations and events found in 
the world around them. 

As stated earlier, the results of this study could promote the creation of a curriculum 
framework for technology education that would emphasize the mental processes involved in 
solving problems in the context of engineering design. A curriculum with this focus would 
occupy a justifiable position in the current era of educational accountability. Recent legislation 
(No Child Left Behind, H.R.366) has suggested that vocational programs be held accountable 
for helping students achieve a blend of academic and vocational goals. Therefore, technology 
education needs to be able to provide substantial evidence of the integration of academic content 
in the curriculum (Oliver, 2004; United States Department of Education, 2005). As a curriculum 
that emphasizes engineering design is implemented in technology education, there will be 
opportunities for the inclusion of concepts from math and science (Hailey et al., 2005). This 
increased rigor in the curriculum will be in line with current educational goals and thinking. 

Limitations of the Study 

The language of the instrument developed for this study will be clear in indicating that 
the study seeks to determine the proper way to implement engineering design as a component 
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of the curriculum in technology education rather than as a part of other content areas. This is a 
critical juncture, and it will be very important to make sure that each participant understands the 
level of specificity expected. Otherwise, the data collected could be very vague and basically 
meaningless. 

There are some areas of uncertainty involved in the study of engineering design in 
technology education. It is not known whether inclusion of engineering principles involving 
math and science will improve student scores on standardized tests in those areas - an important 
consideration in the field of career and technical education. In the future, further research will 
need to be done in order to determine the effect that exposure to this type of curriculum would 
have on students’ standardized test scores. Another area of research which needs to be conducted 
as a follow up to this study might investigate how the actual curriculum delivered in technology 
education classrooms should be structured and organized in order to include the engineering 
design content identified as important by the participants in this study. 

Summary 

In order to create a standards-based curriculum that focuses on helping all students 
achieve technological literacy, engineering design should be incorporated into the technology 
education curriculum (Bearing & Daugherty, 2004; Hailey et ah, 2004; ITEA, 2000; NCETE, 
2004; Wicklein, 2004). Gaining the perspective of recognized experts in the field of engineering 
design is important because that information could assist in the creation of a curricular 
framework for technology education that emphasizes the mental processes involved in problem 
solving in the context of engineering design. This curriculum framework could contribute to the 
ability of professionals in the field to select and implement rigorous curriculum components in 
a learning environment that emphasizes the full range of mental processes involved in solving 
problems. 
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CHAPTER 2 


REVIEW OE LITERATURE 


The history of secondary technology education in the United States is essentially a 
story of change. Manual training was one of the major precursors to industrial arts. In the late 
1800s, John D. Runkle (then president of Massachusetts Institute of Technology) espoused a 
Russian system of education engineers in the practical application of their skills (Wright, 1981). 
He envisioned this system as a part of general education, but his views were not immediately 
accepted. However, another man named Calvin Woodward agreed with Runkle. He found that his 
engineering students were lacking in the ability to mentally picture their designs and were often 
without even rudimentary fabrication skills (Wright, 1981). Woodward was able to establish a 
program based on his beliefs. This system of study was very rigid and required that the students 
progress through a highly supervised course of study that included “predetermined tool exercises 
in wood and metal working” (Herschbach, 1982) along with academic training. Although the 
movement was ultimately short lived, it “was the progenitor of a progression of subjects in the 
US public schools curriculum which has technology as the latest version” (Lewis, 1996, p.l6). 

History indicates that there were many forces creating change during this time and 
another school of thought, originating in Europe, began to take effect: “Sloyd was a second 
system of hand tool education from Europe to be introduced in the United States nearly 
simultaneously with that of manual training” (Smith, 1981, p. 182). Sloyd activities were basic 
and appealed to the creative interests of children (Herschbach, 1982). 

About the same time, the Arts and Crafts Movement was gaining popularity in the 
United States. As part of this movement, “Subjects included in the arts and crafts studies 
included drawing, wood carving, clay modeling, mosaic work, leather carving, metal embossing, 
embroidery, carpentry, wood turning, wood inlaying, and fret sawing” (Smith, 1981, p. 184). 
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Also in the late 1800s a man named Charles Bennett began to advocate what he termed “manual 
arts.” This movement primarily targeted the elementary level; the main legacy it left for the 
development of the field was an emphasis on “integrating drawing and design with construction 
activities” (Smith, 1981, p. 187). Manual training teachers were infiuenced by this movement 
to teach good design in their programs while, at the same time, attempting to foster the creative 
capacities of their students (Barella, 1981). 

Vocationalism had a tremendous infiuence on manual training during this time. In the 
years surrounding the turn of the century, Americans were increasingly aware of the need to give 
students specific industrial skills in order to foster a more advanced society. As Barella (1981) 
notes, “Thus, the cries for secondary school vocational education that fell on deaf ears prior to 
1900 began to intensify and be heard as a new century began” (p. 150) Manual training began 
to evolve into vocational education. Vocational education gradually displaced manual training 
even though many infiuential arguments were made for situating the study of occupations within 
a broader context. Dewey in particular “supported study through occupations, by all students; 
not just those whose probable destinies were in the trades” (Gregson, 1995, p. 7). Prosser and 
Snedden, on the other hand, saw vocational education as a way to prepare the masses for their 
future as workers in need of specific work training. They chose Taylor’s theories on scientific 
management as a pattern for education which was intended to serve the needs of industry 
(Gregson, 1995). 

Those in favor of creating a vocational education that served industry began to take steps 
to create legislation in their favor. During this time there was a tremendous infiux of students 
into the educational system, and new schools opened on a regular basis. Between 1900 and 1917, 
“more than 30 bills introduced in Congress had implications for vocational education” (Hillison, 
1995, p. 4). By 1917 a diverse coalition of organizations had come together to help craft the 
Smith-Hughes Act, which effectively established a formal vocational education supported by 
the Federal government (Rojewski, 2001). This bill only served to exacerbate the differences 
between academic and vocational education that already existed (Kinchloe, 1999). This 
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definition was based primarily on the views of Prosser, who considered separately administered 
and narrowly focused vocational training the best available way to help non-academic student 
secure employment after completing school (Hyslop, 2000). Industrial arts gradually replaced 
manual training in the later stages of schooling (Lewis, 1996). 

These major events shaped the creation of what came to be known as “industrial arts.” 
However, the debate over the focus for industrial arts continued through the years. Various 
curriculum development initiatives were undertaken in an attempt to further the focus and 
define the field. During 1946-47, William Warner and a group of his students at Ohio State 
University undertook a major curriculum development project which they called “A Curriculum 
to Refiect Technology” (Phillips, 1994). This work divided the study of industrial arts into 
the subcategories of communication, construction, power, transportation, and manufacturing 
(Phillips & Lefor, 2002). Curriculum development in the 1960s saw course development 
focus on these specific areas (Phillips, 1994), and Warner went on to become one of the most 
infiuential figures in industrial arts (Latimer, 1982). 

Although Warner’s ideas were not immediately accepted, they did come to fruition in 
later federally funded projects such as the American Industry Project, The Maryland Plan, and 
the Industrial Arts Curriculum Project (Lewis, 1995). There were intermediate steps along the 
way as different groups embraced various schools of thought related to vocational education. 
From the 1950s through the 1970s, industrial arts programs were developed around one of three 
areas: 1 . industry - as exemplified by the Industrial Arts Curriculum Project and the American 
Industry Project; 2. Technology - as promulgated by Olson and Devore; 3. The needs of the child 
- as found in Maley’s work (Wright, 1995). 

These various curriculum approaches existed simultaneously during the 1960s and 1970s 
(Lauda, 2002), but a major step toward creating a unified focus for the field was the release of 
the Jackson’s Mill Industrial Arts Curriculum Theory (Sydner & Hales, 1981), which was a 
result of the Jackson’s Mill Project (Foster & Wright, 1996). Ultimately this work is associated 
with the acceptance of a universal systems model (input, process, output, feedback) and four 
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major systems: transportation, construction, manufacturing, and communication (Lauda, 2002). 
These became part of later technology education curriculum (ITEA, 1996). This project was an 
important development that was influential in conceptualization of technology education (Foster, 
1994). 

Recent Technology Education Literature 

The close ties between technology and engineering are reflected in the literature of 
technology education. Literature in the field has indicated support for including engineering 
content in the technology education curriculum. Bensen and Bensen (1993) stated, “it is 
imperative that we engage the engineering profession. The companies that employ them, 
the universities that educate them, and the associations and accreditation bodies that set the 
standards and benchmarks for them, to become involved in bringing the [technology education] 
curriculum into the twenty-first century” (p. 5). Foster (1996) asked selected leaders in the field 
to choose the most appropriate curriculum approaches in technology education. The participants 
in this study picked engineering systems as the fourth most appropriate approach to technology 
education behind the top three choices, which were math/science/technology integration, design/ 
problem solving, and tech prep. It could be argued that three of these four top choices made by 
the leaders selected in Foster’s study are closely related to engineering. As Faporte observes, 
“Activities that integrate technology, science, and mathematics are essentially engineering 
activities, which are inherently laboratory based investigations with which technology teachers 
are quite comfortable” (1995, p. 184). 

More direct evidence of the intentional inclusion of engineering content within the field 
of technology education comes from The Curriculum Framework for Technology Education 
published by the Georgia Department of Education (2001). This document includes course 
descriptions for the state-approved courses in the area of technology education. At least two 
of these classes are directly related to the topic of engineering, as evidenced by their titles: 
Pre-Engineering Technology (Course number 21.471) and Engineering Applications (Course 
number 21.472). The course description for Pre-Engineering Technology includes the following 
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statements: “This course is designed to introduce students to the concepts and practices of 
engineering technology and engineering careers. Students explore engineering problem solving 
with the integration of mathematics, science, and technology in pre-engineering activities” 
(Georgia Department of Education, 2001, p. 49). Clearly, the focus of this course should be 
the actual practices and activities of the engineering profession. The course description for 
Engineering Applications states that “This course is designed to address three tenets that apply to 
candidates for any engineering program: Students should have a well-rounded base of knowledge 
in as many areas of technology as possible. Students should have an area of specialized interest 
in which they have done extensive work. High school students should have exceptional 
communication skills and be able to make presentations to their peers” (Ga. DOE, 2001, p. 52). 
One of the key elements of the intent of this course is that candidates for engineering programs 
should gain skills that would benefit them in their postsecondary educational endeavors. 

The Technology For All Americans Project 

As the field of technology education has developed, significant strides have been made 
in achieving a unified and well-articulated curriculum focus. Perhaps the most important long- 
term effort directly focused on technology education has been the Technology for All Americans 
Project. This project has been supervised and conducted by the International Technology 
Education Association (ITEA), with funding from the National Science Foundation and NASA. 
Several major publications have been produced as a result of the efforts of this collaboration 
among educators at various levels as well as content experts. This endeavor was very large in 
scope. It emphasized the importance of technology literacy for citizens of increasing complex 
societies and advocated the study of technology for all children. The major publications of 
this project have been Technology For All Americans: A Rationale and Structure for the Study 
of Technology (ITEA, 1996), Standards for Technological Literacy: Content for the Study 
of Technology (ITEA, 2000), and Advancing Excellence in Technological Literacy: Student 
Assessment, Professional Development, and Program Standards (ITEA, 2003). These documents 
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were the major thrust of the Technology for All Americans Project and sought to provide 
direction and continuity to the overall effort of promoting technological literacy. 

Participants in this effort were from an eclectic background including math, science, 
engineering, and technology education. This broad range of expertise and focus is reflected in the 
documents themselves; the connections between the subject areas are emphasized and the ability 
to apply content from various areas to real world problems is a key component of technological 
literacy. It is important to note that the ties between engineering, science, math, and technology 
were emphasized in these publications. 

Technology For All Americans: A Rationale For the Study of Technology 

The Technology for All Americans (ITEA, 1996) document was divided into three 
main parts: the need for technological literacy, the universals of technology, and integrating 
technology into the curriculum. The overall goal of the publication was to provide educators 
and administrators with a clear picture of the technologically literate person and how curriculum 
could reflect an emphasis on the importance of technological literacy. 

The section on the Universals of Technology is particularly important to note because it 
affirmed the broad nature of the study of technology and provided an overall rationale on which 
to base future developments such as the Standards for Technological Literacy. These Universals 
of Technology were defined as processes, knowledge, and contexts. 

The processes of technology were defined as “those actions that people undertake to 
create, invent, design, transform, produce, control, maintain, and use products or systems. The 
processes include the human activities of designing and developing technological systems; 
determining and controlling the behavior of technological systems; utilizing technological 
systems; and assessing the impacts and consequences of technological systems” (ITEA, 1996, 
p. 17). A key aspect of technological literacy is the ability to design and develop technological 
systems “through experiences in designing, modeling, testing, troubleshooting, observing, 
analyzing, and investigating” (p. 18). This emphasis on designing technological systems was an 
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intentional effort to make sure that the proper understanding of technologieal literacy included 
not only information about technology and technological devices, but abilities to creatively solve 
problems using a systems approach. 

Another aspect of the processes of technology is the ability to determine and control the 
behavior of systems. A proper understanding of a system should allow the technologically literate 
person to adjust and control the overall function to achieve the desired output or result: “Analysis 
is required in order to determine how many systems work. Analysis often uses information 
from science and mathematics” in the process of solving problems (ITEA, 1996, p. 19). A 
technologically literate person is able to make use of knowledge and skills from other disciplines 
when dealing with the behavior of systems. 

The second universal of technology was defined as knowledge. This “includes the nature 
and evolution of technology; linkages based on impacts, consequences, resources, and other 
fields; and technological concepts and principles” (ITEA, 1996, p. 16). The technologically 
literate person should recognize that technological systems are developed and understood in 
the context of knowledge from other disciplines and that “technology has a particularly strong 
relationship with science and mathematics” (ITEA, 1996, p. 28). Technology, mathematics, and 
science are interrelated and dependent on each other. Technological literacy emphasizes the 
importance of these relationships and makes use of knowledge from each of these disciplines. 

After providing this framework. Technology for All Americans (ITEA, 1996) discussed 
teaching technology at various educational levels. After discussing the elementary and middle 
school levels, it included an important section on teaching technology at the high school level 
and beyond: 

Technology Education students should evaluate technology’s capabilities, uses and 
consequences on individuals, society, and the environment, employ the resources of 
technology to analyze the behavior of technological systems, apply design concepts to 
solve problems and extend human capability, apply scientific principles, engineering 


18 


concepts, and technological systems in the solution of everyday problems, and develop 

personal abilities related to eareers in teehnology. (ITEA, 1996, p. 40) 

Also, these aetivities should be carried out in the eontext of speeifie technology courses sueh as 
Introduction to Engineering (ITEA, 1996, p. 40). Clearly, this document intended to embraee the 
field of engineering and make it an important part of technologieal literacy for students. 
Standards for Technological Literacy 

The Teehnology for All Amerieans Project also produced the Standards for 
Technological Literacy (ITEA, 2000). This important publieation built on earlier work related 
to technologieal literacy. The purpose was to provide “a vision of what students should know 
and be able to do in order to be teehnologically literate” (ITEA, 2000, p. 7). This vision was 
artieulated in the form of twenty content standards that outlined basie curricular goals for 
technology education at the K-2, 3-5, 6-8, and 9-12 grade levels. These standards were intended 
to shape the curriculum of technology education in order to help students achieve teehnological 
literaey. The standards “are statements about what is valued and can be used for making a 
judgment of quality” (ITEA, 2003, p. 7) regarding the curriculum chosen for technology 
education programs. The Standards for Teehnological Literacy are divided into five sections: 

(1) The Nature of Technology, (2) Technology and Society, (3) Design, (4) Abilities for a 
Technologieal World, and (5) The Designed World (ITEA, 2003). These broad categories help to 
define the eriteria that make up technological literacy. 

Several of the standards have direet applieation to this study and are included here: 
Standard 8: Students will develop an understanding of design; 

Standard 9: Students will develop an understanding of engineering design; 

Standard 11: Students will develop the ability to apply the design process (ITEA, 2000). 

The elear argument of this publieation is that aetivities in teehnology education are intended 
to teach skills necessary in professions sueh as engineering. Lewis (2000) indicated that the 
design called for in the STL is aetually the same proeess employed by engineers when they solve 
problems: 
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Students in technology laboratory-classrooms are taught practical problem solving skills 
and are asked to put them to work on different types of real-world problems. Engineers, 
architects, computer scientists, technicians, and others involved in technology use a 
variety of approaches to problem solving, including trouble shooting, research and 
development, invention, innovation, and experimentation. Students will become familiar 
with these approaches and learn about the appropriate situations in which to use them. 
(ITEA, 2000, p. 5) 

In explaining Standard 8, the document gives further detail about the design process by stating 
“the engineering profession has developed well-tested sets of rules and design principles that 
provide a systematic approach to design. Design measurability, which is a key concept of the 
engineering profession today, is concerned with a designer’s ability to quantify the design 
process in order to improve the eflhciency” of the system (ITEA, 2000, p. 91). The emphasis 
is on creating situations within the K-I2 environment in which students have the opportunity 
to gain real-world skills such as “performing measurements, making estimates and doing 
calculations using a variety of tools, working with two-and three-dimensional models, presenting 
complex ideas clearly, and devising workable solutions to problems” (ITEA, 2000, p. 90). 

The ninth standard deals with engineering design specifically. While recognizing that 
the field of engineering has not come to a consensus on a single definition of what actually 
happens in engineering design, it is clear that students should be familiar with: identifying the 
problem, generating ideas, selecting possible solutions, evaluating solutions (often through 
models and prototypes), and refining solutions, and implementing the solution (ITEA, 2000, 
p. 99). This standard calls for a broad understanding of the nature of engineering design and 
the many factors that go into the process such as “safety, reliability, economic considerations, 
quality control, environmental concerns, manufacturability, maintenance and repair, and human 
factors engineering (ergonomics)” (ITEA, 2000, p. 105). The process also involves personal 
characteristics that should be emphasized by educators “such as creativity, resourcefulness, and 
the ability to visualize and think abstractly” (p. 104). 
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The eleventh standard deals with students’ ability to apply the design proeess. This 
proeess is responsible for the development of most technologies: “Very few products or systems 
today are developed by trial and error or come by accident. Instead, almost any technology that a 
student encounters is the result of a “systematic problem-solving design process that transformed 
an idea into a final product or system” (ITEA, 2000, p. 115). This publication emphasizes the 
importance of understanding the various components of design so that students can give clear 
explanation of the choices made in reaching solutions to design problems. Secondary students 
should be given challenging problems that require a synthesis of knowledge from various 
disciplines so that they achieve the greatest level of learning possible. Students should be 
required to 

Refine a design by using prototypes and modeling to ensure quality, efficiency, and 
productivity of the final product. Evaluate the design solution using conceptual, physical, 
and mathematical models at various intervals of the design process in order to check for 
proper design and to note areas where improvements are needed. Develop and produce 
a product or system using a design process. Evaluate final solutions and communicate 
observation, processes, and results of the entire design process, using verbal, graphic, 
quantitative, virtual, and written means, in addition to three-dimensional models, (p. 124) 
Applying the design process is a crucial juncture in achieving technological literacy because it 
is here that students need to have the ability to draw upon principles from other disciplines in 
the creation, evaluation, and analysis of their designs. Specifically, students should be required 
to explain their decisions using verbal and quantitative skills transferred from language arts and 
mathematics. The solutions to technological problems that students design should also be based 
on basic scientific principles that can be reinforced in the technology education classroom and 
applied in the design process. 

Gorham (2002) noted distinct similarities between criteria selected by the Accreditation 
Board for Engineering and Technology (ABET) and the Standards for Technological Literacy. 
The ABET “is widely recognized as the agency responsible for accrediting educational programs 


21 


leading to degrees in engineering” (Gorham, 2002, p. 30). Ineluded below are the eleven eriteria 
selected by this agency as mandatory for all engineering preparatory programs. It is especially 
important to note the similarities to the goals and standards for technological literacy: 

Criterion A: Ability to apply knowledge of mathematics, science, and engineering 
Criterion B: Ability to design and conduct experiments, as well as to analyze and 
interpret data 

Criterion C: Ability to design a system, component, or process to meet desired needs 

Criterion D: Ability to function on multi-disciplinary teams 

Criterion E: Ability to identify, formulate, and solve engineering problems 

Criterion F: Understanding of professional and ethical responsibility 

Criterion G: Ability to communicate effectively 

Criterion H: Broad education necessary to understand the impact of engineering solutions 
in a global and societal context 

Criterion I: Recognition of the need for and an ability to engage in lifelong learning 

Criterion J: Knowledge of contemporary issues 

Criterion K: Ability to use techniques, skills, and modern engineering tools necessary for 
engineering practice (ABET, 2005, p.6) 

Gorham (2002) displayed similarities between the Standards for Technology Eiteracy 
(STL) and the ABET criteria. Each of the 11 criteria selected by ABET were matched to 
corresponding standards and the results showed extremely strong correlations. Ten of the eleven 
ABET criteria were matched to at least one of the STL standards with most having correlation to 
multiple standards. Gorham concluded that “as school districts adopt and implement Standards 
for Technological Literacy, increased numbers of pre-college students will be exposed to the 
breadth of engineering” (p. 30). 
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Advancing Excellence in Technological Literacy: Student Assessment, 

Professional Development, and Program Standards 

In 2003, the International Technology Education Association published a follow 
up document cslWqA. Advancing Excellence in Technological Literacy: Student Assessment, 
Professional Development, and Program Standards (ITEA, 2003). This was also part of 
the Technology for All Americans Project. Its goal was to further explain how teachers, 
administrators, and others could participate in the goal of technological literacy for all students. 
This publication, also referred to as AETE, sought to reinforce the groundwork established by the 
Standards for Technological Literacy and broaden the perspective to include administrators and 
others in positions to make decisions about technology education programs. 

It is important to note that AETE also included references to the close ties that exist 
between technology and other fields such as mathematics, science, and engineering. It also 
encourages teachers to help students understand these connections and to make use of subject 
matter from these disciplines in the study of technology. In addition, teachers “may require 
students to identify technological problems, needs, and opportunities within a cultural context; 
write and construct problem statements; design, develop, model, test, prototype and implement 
solutions; analyze, evaluate, refine, and redesign solutions; and refiect and assign value to 
processes and outcomes” (ITEA, 2003, p. 31). Solving problems in this context calls for a broad 
understanding of the problem-solving process and the use of information and skills from a 
variety of sources. 

One of the goals of AETE is to help teachers understand how to assess students as 
teachers develop technological literacy. Various methods of formative and summative assessment 
are discussed. Standard A-4 makes it evident that assessment should occur in the context of 
how students actually learn technology rather than only traditional methods of pencil and 
paper testing. Assessment should be an informative process; it should allow the teacher to truly 
gauge the students’ understanding of the content and should include multiple means that are all 
authentic. Part of this authentic assessment should involve tasks that require students to “use 
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appropriate technology, science, and mathematic principles” (ITEA, 2003, p. 32) in their projects 
so that the level of technological literacy can be demonstrated. The AETL emphasizes the role of 
engineering design in technology education: 

There are strong philosophical connections between the disciplines of technology and 
engineering. The engineering profession has begun to work with technology teachers to 
develop alliances for infusing engineering concepts into K-12 education. The alliances 
will provide a mechanism for greater appreciation and understanding of engineering and 
technology. The National Academy of Engineering is as avid supporter of technological 
literacy, (p. 13) 

In order for teachers to be able to adequately teach and assess learning in this context, the AETL 
document provides standards related to professional development detailing the skills educators 
need. Teachers should be constantly learning and enhancing their knowledge of technology and 
its relationship to math, science, and engineering (ITEA, 2003) in order to help students achieve 
technological literacy. Part of the strategy for accomplishing this goal is for teachers to know the 
process of design and to be “acquainted with engineering design” (ITEA, 2003, p. 42) as well as 
other problem-solving strategies. 

Related Literature 

Technically Speaking: Why All Americans Need to Know More About Technology 

These documents from the Technology for All Americans Project are catalysts in 
achieving technological literacy. Another very important recent publication has been a joint 
publication from the National Academy of Engineering and the National Research Council 
titled Technically Speaking: Why All Americans Need to Know More About Technology (Pearson 
& Young, 2002). This publication was a direct result of a two-year effort by the Committee 
on Technological Literacy, which operated under the direction of the National Academy of 
Engineering. The dominant theme of the text is achieving technological literacy in our country 
through a variety of formal and informal means. This publication discussed the major reasons 
for focusing on technological literacy, the benefits students would receive, the proper context. 
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and the philosophical foundation underlying this initiative. Does this include a call for the study 
of engineering design? Engineering design is given an important place in technological literacy 
and even state that the one of the characteristics of a technologically literate person is that such 
a person is “familiar with the nature and limitations of the engineering design process” (Pearson 
& Young, 2002, p. 17). Technological literacy is viewed as a requirement for all students and 
a national concern that merits the best efforts of all those having influence over formal and 
informal educational settings. Pearson and Young discuss the lack of emphasis education about 
technology has received in recent years and also ways to overcome these deflciencies. 

Technically Speaking also directly addresses those who teach technology and 
recommends that technology teachers “approach the subject from an engineering perspective 
rather than an industrial arts perspective” (Pearson & Young, 2002, p. 108). Technology 
educators are encouraged to become familiar with the interrelationship between technology and 
other subjects, especially science and math, and to help other teachers integrate the study of 
technology into their curriculum. The design process plays a prominent role in the instructional 
strategies recommended by Technically Speaking. According to a recent work, “teachers at all 
levels should be able to conduct design projects and use design-oriented teaching strategies to 
encourage leaming”(Pearson & Young, 2002, p. 108). 

For all the indicators pointing to the inclusion of engineering content in the technology 
education curriculum, there is actually a lack of research on how to go about this process. In a 
recent survey of high school technology education teachers, Wicklein and Gattie (2004) found 
that the 279 respondents had a generally positive view of the value of engineering design 
for technology education. In fact, an average of 90.6% of the survey respondents indicated 
agreement or strong agreement with the fourteen statements on the beneflts of including 
engineering design in the technology education curriculum. Although the survey indicated 
consistently positive responses regarding the perceived value of engineering, there was also 
agreement on the need for additional assistance in developing an appropriate curriculum. The 
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following percentages reflect the number of those who either indicated agreement or strong 
agreement with the statements: 

My instructional needs to teach engineering design include: 

identifying appropriate instructional content (91.4%), determining the appropriate level of 
instruction (89.7%), integrating the appropriate levels of mathematics and science into the 
instructional content (93.8%), having the appropriate types of tools and test equipment to teach 
engineering design (93%), developing additional analytic (math) skills to be able to predict 
engineering results (87.3%), improving fundamental knowledge of engineering sciences (statics, 
fluid mechanics, dynamics) (91.9%). 

Engineering Design 

Engineering has been called “a fundamental human process that has been practiced from 
the earliest days of civilization” (Petroski, 1996, p. 2). From ancient times men have endeavored 
to use tools and processes to meet their needs and to make life more tolerable. Human need 
served as the catalyst for the application of knowledge and resources to the problem at hand. The 
word “engineer” can be traced to Roman times “when the Latin expression ingenium was used to 
suggest some ingenious attribute of an object or a person. Eventually, the derivative ingeniator 
was applied to a person possessing an innovative mind and skillful hands in the making of such 
devices” or products (Harms et ah, 2004, p. 3). Over time, the methods used to solve problems 
and make ingenious devices coalesced into a series of steps that have come to be known as the 
engineering design process. Engineering literature defines and explains this process, although 
there is some debate about the exact nature of the process and the exact steps involved. 

Some of the definitions of engineering design in the literature are succinct and extremely 
broad: “Engineering design is a systematic process by which solutions to the needs of humankind 
are obtained” (Eide et al., 2002, p. 79). Another one is “Engineering design is the systematic, 
intelligent generation and evaluation of specifications for artifacts whose form and function 
achieve stated objectives and satisfy specified constraints” (Dym, 1994, p. 17). This process 
has been described as being “as varied as the engineering profession, and it is as broad as the 
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problems facing humankind” (Wright, 2002, p. 111). Other definitions are more complete and 
describe this process in greater depth: 

The accreditation board for Engineering and Technology (ABET) has traditionally defined 
engineering design as follows: Engineering design is the process of devising a system, 
component, or process to meet desired needs. It is a decision making process in which the 
basic sciences and mathematics and engineering sciences are applied to convert resources 
optimally to meet a stated objective. Among the fundamental elements of the design 
process are the establishment of objectives and criteria, synthesis, analysis, construction, 
testing, and evaluation. It is essential to include a variety of realistic constraints, such as 
economic factors, safety, reliability, aesthetics, ethics, and social impact. (Oakes et ah, 

2002, p. 339) 

In addition to these definitions, the engineering design process is usually thought of as a series of 
steps. Various authors have defined the steps differently. Wright identified the following steps: 

1 . Identification of the problem 

2. Gathering needed information 

3. Searching for creative solutions 

4. Stepping from ideation to preliminary designs (including modeling) 

5. Evaluation and selection of preferred solution 

6. Preparation of reports, plans, and specifications 

7. Implementation of the design (Wright, 2002, p. 113) 

These basic steps are repeated in the literature of engineering (with variations) and are generally 
typical of the descriptions given. Some descriptions include more steps because of greater detail 
in their explanation or because of breaking some parts into more than one step. One example of 
this situation is the ten-step design process (Eide et ah, 2002, p. 81): 

1 . Identification of need 

2. Problem definition 

3. Search 
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4. Constraints 


5. Criteria 

6. Alternative Solutions 

7. Analysis 

8. Deeision 

9. Specifications 

1 0. Communication 

This process is sometimes depicted as a circle or a repeating cycle in order to emphasize the 
iterative nature of the process and to reinforce the concept of continual improvement of possible 
solutions. French (1992) described the process in a fiow chart (see Figure 1) that provides greater 
detail on the order of the steps and iterative nature of the design process. 
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Figure 1 

French’s design process diagram 



Feedback 
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An engineering eurrieulum developed by the Massachusetts Department of Education describes 
the engineering design process as follows: 

1 . Identify the need or problem 

2. Research the need or problem 

Examine current state of the issue and current solutions 
Explore other options via the Internet, library, interviews, etc. 

3. Develop possible solution(s) 

Brainstorm possible solutions 
Draw on mathematics and science 

Articulate the possible solutions in two and three dimensions 
Refine the possible solutions 

4. Select the best possible solution(s) 

Determine which solution(s) best meet(s) the original requirements 

5. Construct a prototype 

Model the selected solution(s) in two and three dimensions 

6. Test and evaluate the solution(s) 

Does it work? 

Does it meet the original design constraints? 

7. Communicate the solution(s) 

Make an engineering presentation that includes a discussion of how the 
solution(s) best meet(s) the needs of the initial problem, opportunity, or need 
Discuss societal impact and tradeoffs of the solution(s) 

8. Redesign 

Overhaul the solution(s) based on information gathered during the tests and 
presentation (Massachusetts DOE, 2001) 

Regardless of the actual set of steps chosen or the graphic model used to depict the 
various aspects of the design process, there are certain tasks common to the engineering design 
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process known as synthesis, analysis, and evaluation. These tasks are repeated at various stages 
in the process. Dym (1994) says that “Synthesis is the task of assembling a set of primitive 
design elements or partial designs into one or more configurations that clearly and obviously 
satisfy a few key objectives and constraints. Synthesis is often considered as the task most 
emblematic of the design process”(p. 28). As the term implies, the task of synthesis frequently 
“involves combining facts, principles, or laws into a whole idea” (Wright, 2002, p. 118). 
Synthesis in engineering includes the eoncept of relating ideas to one another in such a way as to 
create a solution to a problem based on the interrelationship of the various parts in a systematic 
solution (Buhl, 1960). 

Another task central to engineering design is analysis. In the process of analysis, 
engineers “break down accumulated information to determine, item by item, its contribution 
to the whole problem” (Buhl, 1960, p. 76). The goal is to determine the characteristics and 
resulting infiuence of each individual component of a possible solution. Analysis allows 
engineers to “work with the governing equations and relationships” (Burghardt, 1999, p. 75) that 
are necessary for a true understanding of the problem at hand. Dym notes that “Analysis is the 
task of performing those calculations (or analyses) needed to assess the behavior of the current 
synthesis - or embodiment or preliminary design” (1994, p. 28). 

The third task assoeiated with engineering design is the evaluation task. This task occurs 
when “we compare our analyses of the attributes and behavior of the current design to the stated 
design specifications and constraints to see if this synthesis is acceptable” (Dym, 1994, p. 28). In 
other words, the task questions the appropriateness of the solution obtained by analysis. 

Aeeountability 

In the current era of educational reform, career and technical education is being held 
responsible for providing programs that enable students to reach academic accountability. 

Student success is currently one of the requirements being asked of all programs, including 
career and technical education (Green, Stacey, & Tulley, 2005). Current legislation affecting CTE 
has continued the trend toward a emphasis on academic achievement for all students: 
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In the decade of the 1990s, the federal government passed three laws that were intended 
to, among other outcomes; improve the academic performance of high school youth who 
“majored” in CTE. The Carl D. Perkins Vocational and Applied Technology Education 
Act of 1984 revisions in 1990 (Perkins II) and 1998 (Perkins III) reflected a philosophical 
shift in the goal of vocational education (or CTE), from a narrow focus on occupational 
preparation for special populations to a more academically rigorous program that 
prepared students for participation in industry as well as for postsecondary education. 
(Stone, 2004, p.53) 

Secondary level technology education programs exist in the larger educational setting of career 
and technical education. Hoachlander (1998) noted that a curriculum framework reflective of the 
current state of career and technical education should “help strengthen the academic foundation 
of secondary and postsecondary education by helping students learn and apply a wide range of 
academics in a work-related context” (p. 4). As Delci and Stem (1999) have commented. 
Conflicting crosscurrents have flowed through vocational education in American 
secondary schools during the 1980s and 1990s. Overall enrollment in vocational courses 
has fallen. But against this ebbing tide, an incoming current has brought a growing 
number of participants into new programs and curricula. While traditional vocational 
offerings have been geared toward immediate entry into speciflc occupations, new 
programs and course sequences are intended to prepare students for both college and 
careers, by combining a challenging academic curriculum with development of work- 
related knowledge and skill, (p. 9) 

Stone (2004) found that CTE students were taking a more rigorous academic content. This has 
been demonstrated by the increased number and difficulty of the math courses taken. 

Mental Processes 

Custer (1995) distinguished between technological problem solving and other forms 
of solving problems. Technological problem solving has been identified as the major element 
involved in the processes of technology that have been identified in the literature of technology 
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education (Hill & Wicklein, 1999). Technological problem solving relies on mental processes 
identified by Halfin (1973). Halfin’s doctoral study identified 17 mental processes (see Table 4). 


Table 4 

Mental Processes Identified by Halfin 


Analyzing 

Measuring 

Communicating 

Modeling 

Computing 

Models/prototypes 

Creating 

Observing 

Defining problem(s) 

Predicting 

Designing 

Questions/hypotheses 

Experimenting 

Testing 

Interpreting data 
Managing 

Visualizing 


Wicklein and Rojewski (1998) reaffirmed Halfin’s original mental processes and extended the list 
to include several additional processes considered vital to the solving of technological problems 
(see Table 5). 


33 


Table 5 


Mental Processes Identified by Delphi 


Analyzing 

Communicating 

Computing 

Contexts 

Creating 

Customer Analysis 
Defining Problem(s) 
Designing 
Establishing Need 
Experimenting 
Innovating 
Interpreting Data 
Managing 
Measuring 


Modeling 
Models/Prototypes 
Monitoring Data 
Observing 
Predicting 

Questions/Hypotheses 
Researching 
Searching for Solutions 
Technology Review 
Testing 

Transfer/Transformation 

Values 

Visualizing 


This list of processes was condensed into a more manageable list of key constructs (Hill & 
Wicklein, 1999) associated with technological problem solving (see Table 6). 


Table 6 

Mental Processes Identified by Factor Analysis (Hill & Wickein, 1999). 


researching 

managing 

creating 

computing 

monitoring data 

questions/ 

hypotheses 

measuring 

defining 

problem(s) 

predicting 

models/ 

prototypes 

values 

technology 

review 

innovating 

visualizing 

observing 

establishing need 

communicating 

customer 

analysis 

modeling 
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CHAPTER 3 


METHOD 


This chapter provides a description of the research design, sample, instrument, and 
procedures that will be used to conduct this study. In addition, this section contains a discussion 
of how data was analyzed. 

Research Design 

This study relied on input from experts in the field of engineering regarding the nature 
of the engineering design process and how it should be taught to secondary students enrolled in 
Technology Education classes. An expert is “a person who has background in the subject area 
and is recognized by his peers or those conducting the study as qualified to answer questions” 
(Meyers & Booker, 1990, p. 3). Expert judgment is frequently used to make predictions about 
future events and has been defined as “the assertion of a conclusion based on evidence or 
an expectation for the future, derived from information and logic by an individual who has 
extraordinary familiarity with the subject at hand” (Millet & Honton, 1991, p. 43). Meyers and 
Booker (1990) identified several situations when expert judgment is typically gathered: 

- To provide estimates on new, rare, complex, or otherwise poorly understood 
phenomena 

- To forecast future events 

- To integrate or interpret existing data 

- To learn an expert’s problem-solving proeess or a group’s decision making processes 

- To determine what is currently known, what is not known, and what is worth learning 
in a field of knowledge, (pp. 4-5) 

Research that makes use of the judgment of a panel or group of experts has been shown to be 
valuable and accurate (Brown & Helmer, 1964). One reason for this outcome is that “the total 
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information available to the group is probably many times that possessed by any single member” 
(Martino, 1983, p. 14). 

Expert judgment is frequently gathered in a form that is quantitative in nature (Bookers 
& Meyers, 1990). This data can be considered to be formal and “involves selecting experts 
according to particular criteria, designing elicitation methods, and specifying the mode in which 
the expert is to respond” (Meyers & Booker, 1990, p. 6). This study used the Delphi method 
because it allows experts to have input on the topic of this study in a very efficient manner. The 
basic idea of the Delphi procedure is “repeated administration of questionnaires to each member 
of an expert panel, without face-to-face contact” (Dean & West, 1999, p. 4). 

The original development of this method has been attributed to a Rand Corporation study 
for the United States military in the 1950s (Dalkey & Helmer, 1963), and it has come to be a 
common research methodology in various fields, especially when consensus on a topic is desired 
(Borg & Gall, 2003). The Delphi method has been noted as an effective means of facilitating 
a group decision process on a complex subject or problem (Linstone & Murray, 1975). This 
method of research is extremely fiexible and lends itself to a broad range of applications. The 
Delphi method has been described as “a method of eliciting and refining group judgments” 
(Dalkey, 1969, p. 5). The Delphi method has distinct advantages that are documented in the 
literature: 

1 . The consensus reflects reasoned, self-aware opinions, expressed in the light of the 
opinions of associate experts. Thus, these predictions should provide a sounder basis 
for long-range decision making than do unarticulated intuitive judgments. 

2. Research suggests that face-to-face discussion tends to make the group estimates less 
accurate, whereas the controlled-feedback procedure makes group estimates more 
accurate. 

3. The procedures create a well-defined process that can be described quantitatively. 
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4. A meaningful estimate of the accuracy of a group response to a given question can 
be obtained by combining individual self-ratings on that question into a group rating. 
(Lanford, 1972, p. 22) 

There are three important features of the Delphi method that are useful for this study: 
anonymous response, iteration and controlled feedback, and statistical group response (Dalkey, 
1969): 

1. Anonymous response - opinions of the group are obtained by formal questionnaire. 

2. Iteration and controlled feedback - interaction is effected by a systematic exercise 
conducted in several iterations, with careful, controlled feedback between rounds. 

3. Statistical group response - the group opinion is defined as an appropriate aggregate 
of individual opinions on the final round. These features are designed to minimize 
the biasing effects of dominant individuals, of irrelevant communications, and group 
pressure toward conformity, (p. 5). 

Anonymous response is important to allow individuals to freely express their true 
opinions on matters related to the subject at hand. Anonymity is also important because 
individuals of higher status may exert too great an infiuence on other group members. Also, 
group members who perceive of themselves as lower in status may tend to be hesitant to offer 
their ideas (Turoff & Hiltz, 2005). Dalkey (1969) found that anonymous response had a tendency 
to make the overall assessment more accurate. 

The fact that the study has iterations with controlled feedback is extremely important 
because this approach helps facilitate and clarify all communication; the experts involved in the 
study will have the opportunity to receive feedback on the responses given by other members of 
the group and then to clarify or add to their original answer in the second iteration of the study. 
The ability to achieve a true statistical group response is also very important and minimizes the 
chance of having a committee swayed inordinately by a few dominant personalities (Turoff & 
Hiltz, 2005). 
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There are typically several rounds or iterations involved in a Delphi study. As the 
iterations proceed, there is a general trend toward a greater understanding of the questions 
involved and group consensus (Dalkey, 1969). It is important to note that complete agreement 
on all parts of the problem is not considered a realistic goal and should not be expected by 
researchers. In general, the questionnaires become more specific and focused in later iterations 
because they contain feedback for the participants regarding the groups’ responses to earlier 
questions. 

Population and Sample 

An initial group of engineering design experts was identified through contact with Dr. 
Clive Dym, director of the Engineering Design Center at Harvey Mudd College, Claremont, 
California. Dr. Dym is an internationally recognized expert on engineering design issues and 
has organized and conducted engineering design conferences for the past nine years. These 
conferences typically are attended by internationally recognized experts in engineering design. 

In April of 2006, Dr. Dym was asked (see Appendix A) to identify a panel of 10 persons whom 
he considered to be experts in engineering design who could serve as participants in this study. 
Dr. Dym actually identified 12 persons whom he considered to be highly qualified. The 12 
people thus identified were contacted through email and asked to identify 10 leading experts in 
engineering design each. Ten of the original list of 12 agreed to supply names and generated a 
pool of 59 names. This was a somewhat shorter list than hoped for and due to time constraints it 
was decided to invite each of these persons on this list to participate in the study. 

At this point, this study began to utilize the services of the Hostedsurvev.com website. 
This service was utilized to make initial contact with the potential participants, send reminder 
emails, and conduct each round of the study. With the vast capabilities of the Internet, research 
conducted online has become increasingly popular (Wong, 2003). Typically, “the Internet is used 
as the medium and the exercise and the material are generally posted on a Web site. Participants 
will log onto the Web site during a specific period (e.g., a week) and provide inputs at times 
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compatible with their individual schedules” (p. 18). Participants accessed the survey website to 
view each questionnaire and responded with either a written message or a numerical response 
as the question dictated. Participants had a window of time (10 days) in which to complete the 
current round. On the day the current survey became available online, an email was sent to let 
participants know what set amount of time was available in which to access the website and 
answer the questionnaire. 

During May and June of 2006 contact was made with each of the 59 potential participants 
through email and also through telephone calls when necessary. The initial email (see Appendix 
B) described the study and also contained a link to the website that collected simple demographic 
data and consent to participate in the four-round Delphi study. The number of participants desired 
was 25 because this number would leave room for the possible attrition of some members of 
the panel during the study due to circumstances beyond their control (Martino, 1983). Table 7 
contains the timeline of the study. 


Table 7 

Study Timeline for 2006 (Deadline to join the study - June 16) 


Round # 

Date 

Round 1 

June 19-28 

Round 2 

July 10 - 19 

Round 3 
Round 4 

July 24 - August 2 
August 7-16 


By the June 16, 2006 deadline, 22 people had indicated their willingness to participate 
in the study. Many had to be reminded of this deadline, and dozens of calls were made to the 
potential participants who had not responded to the initial email invitation. Of the 22 people who 
agreed to participate, 21 accessed the website to indicate their agreement, and one person gave 
verbal consent. 
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Round 1 


After the initial deadline to join the study, an email was sent out to all who had agreed to 
participate (hereafter called “participants”). This email contained some brief instructions and also 
a hyperlink which allowed access to the study website (see Appendix C). Participants were asked 
to provide 7-10 phrases or short answers to the four research questions: 

1 . What aspects of the engineering design process best equip secondary students to 
understand, manage, and solve technological problems? 

2. What mathematics concepts related to engineering design should secondary students 
use to understand, manage, and solve technological problems? 

3. What specific science principles related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

4. What specific skills, techniques, and engineering tools related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 

This was a very crucial stage because it was imperative that all participants understood the 
questions correctly so that their responses would be applicable to the research goals of the study. 

A reminder email was sent out three days before the end of Round 1, and telephone calls 
were made to solicit responses. A total of 15 out of the 22 original participants completed the 
Round 1 survey by the deadline of June 28, 2006. Two hundred and thirty-four total responses to 
the four research questions were recorded (see Appendix D). 

After the close of Round 1 on June 28, 2006, it was necessary to conduct a review of 
the data to establish a valid list of all unique responses to the four research questions. Outside 
reviewers were identified by contacting Dr. Kurt Becker at Utah State University and asking 
him to recommend reviewers. Dr. Becker recommended Dr. Paul Schrueders, who in turn 
recommended Dr. Tim Taylor. Consequently, the raw data from Round 1 was compiled and sent 
to Drs. Schrueders and Taylor, both of whom are members of the engineering faculty at Utah 
State University. Dr. Schrueders is in the Department of Engineering and Technology Education, 
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while Dr. Taylor is a member of the Department of Biological and Irrigation Engineering. These 
men agreed to review the raw Round 1 data and create a list of all unique responses to each of 
the four research questions. The data was sent to them on June 29, 2006. The Round 1 data was 
compiled into a list of 88 unique responses and are identified in Table 13 of Chapter Four. 

Round 2 

The review completed by Drs. Schrueders and Taylor allowed the creation of the Round 2 
survey based on their assessment of the Round 1 data. The Round 2 survey (see Appendix E) was 
made available to participants July 10, 2006. On that day an email was sent out to each of the 15 
persons who had completed the Round 1 survey, informing them that the survey was available 
and would be open until July 19, 2006. Each unique response identified from the Round 1 data 
by Drs. Schrueders and Taylor was included in the Round 2 survey. Each item was listed along 
with a 6-point Eikert scale (see Table 8) which allowed the participants to indicate their level of 
agreement or disagreement with each statement. 


Table 8 
Eikert Scale 


Strongly 

Disagree 

Disagree 

Somewhat 

Disagree 

Somewhat 

Agree 

Agree 

Strongly 

Agree 

1 

2 

3 

4 

5 

6 


In addition, participants were asked if there were any additional items that they wished to add 
to the list of responses from Round 1 . Participants were free to add any additional items at this 
point that they felt helped to answer the research questions. 

On July 16, 2006 a reminder email was sent out to all participants who had not completed 
the Round 2 survey. In addition, telephone calls were made to those who had not responded to 
the email invitation or subsequent reminder. This process resulted in 13 responses to the Round 2 
survey by the July 19, 2006 deadline. 
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A statistical analysis of the responses in Round 2 resulted in an empirieal measure of the 
level of support afforded each individual response by the group. The data from Round 2 was 
analyzed using descriptive statistics, and the mean, maximum, minimum, standard deviation, and 
interquartile range were ealeulated. The most important statistie involved in a Delphi study is the 
median response to eaeh item (Dalkey, 1968) because this outcome most aeeurately deseribes 
the overall rating of the particular item. The mean, standard deviation, and interquartile range 
were also used to report on the group response to the various items generated by the partieipants. 
The interquartile range is a common statistical measure denoting the distance between the 75th 
and 25th percentiles. The interquartile range is the middle 50% of the responses to an individual 
item and was the primary measure of the degree of consensus aehieved. A eommon measure of 
the interquartile range that indicated an acceptable level of agreement has been identified as less 
than 1.2 (Custer, Scareella, & Stewart, 1999). For the purposes of this study, the interquartile 
range that indieated a high level of eonsensus was a seore that was < 1 . A range of one (1) or less 
indicated that the middle 50% of responses were either identical (IQR=0) or sequential (IQR=1). 
Helmer (1983) noted that the greatest degree of eonvergenee of expert opinion in Delphi studies 
generally occurs between Rounds 1 and 2. 

Round 3 

The Round 3 survey (see Appendix G) included all the previous rounds plus additional 
items suggested by partieipants in Round 2. Items brought forward from Round 1 thus had 
numeric scores associated with them that had been analyzed as mentioned previously but the few 
new items identified did not. The Round 3 survey was prepared so that the Round 2 statistical 
data for each individual item (except the new ones identified in Round 2) was displayed beside 
the item along with the six-point Likert seale. This presentation was important because each 
participant could compare his own Round 2 seore with the group’s seores. The purpose in Round 
3 was to allow the experts to see how others in the sample group responded in Round 2 and to 
give them a chanee to revise their own responses in light of the group response to the same items. 
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The survey ineluded space for participants to add comments on any of their answers that fell 
outside the interquartile range. As in the Round 2 survey, participants also had the opportunity to 
add any new items if they wished. 

On July 24, 2006 an email was sent to the 13 participants who had completed the 
Round 2 survey, informing them that the survey was available and asking them to finalize their 
responses by August 2, 2006. On July 31a reminder email was sent out and additional telephone 
calls were made to remind participants to complete and submit the survey. A total of 13 surveys 
were submitted by the August 2 deadline. 

Round 4 

The Round 3 data was analyzed using the descriptive statistics mentioned previously. 
Some Round 3 items were omitted from Round 4 because they met the criteria for stability and 
consensus during the first three rounds. The literature finds that the vast majority of participants 
have been found to reach their final conclusions by the third round (Cyphert & Gant, 1971; 
Martino, 1983) and that three rounds are typically enough for the study to reach stability 
(Tinstone & Turoflf, 1975). The degree of stability was gauged by determining the percentage of 
mean change between rounds for each response. Items that had a less than 15% overall change in 
mean score between rounds two and three were considered stable (Scheibe et ah, 1975). Stability 
of the experts’ responses is one of the major indications of the validity of the results (Dalkey, 
1969). 

A second consideration at this point was the level of consensus each item had achieved. 
The same parameter described previously (IQR ^1) was used to determine whether participants 
had reached consensus or not. A large percentage of items from Round 3 had achieved stability 
by this definition. 

The third consideration for inclusion in Round 4 was whether an item had any comments 
from Round 3. A very small number of comments were made defending the score given an item 
in any of the rounds and these were included in subsequent rounds so that all participants were 
aware of the discussion. In addition to the above conditions, there were a few additional items 
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added in Round 3 that were added to the Round 4 survey. They had no statistical data associated 
with them since they were new. Items that were included in the Round 4 survey (see Appendix 
I) had the results from Round 3 listed beside them along with the 6-point Likert scale. Any 
comments submitted in Round 3 were also included with the appropriate item so that it was 
available to all participants. 

Summary 

This study sought to contribute to the development of an understanding of engineering 
design within technology education. This is an important and timely topic because of the recent 
emphasis on engineering and engineering design in the literature of technology education. 

The Delphi method has been identified as an appropriate means of creating a dialogue among 
professionals and making informed decisions based on the expert judgment of persons with a 
great degree of knowledge in a particular field (Meyer & Booker, 1990). Participants in this 
study were identified as experts in the field of engineering design by their peers through a 
process that is logical and repeatable. Twenty-two participants were selected from a list of 59 
persons considered to be highly knowledgeable in the area of engineering design. 

This study utilized four rounds to give multiple opportunities for the group opinion to 
coalesce. Each round of the study was conducted via the Internet and participants accessed the 
questionnaires at a website dedicated to this study. Participants had a set amount of time in which 
to record their answers on the web-based questionnaires that utilized Likert scales. Descriptive 
statistics were used to report the group responses and to ascertain the degree of stability for each 
item. The panel of experts anonymously participated in the study. All items in the study will be 
included in the final report, whether or not they achieved consensus and stability. 
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CHAPTER 4 


RESULTS 


A four-round Delphi research process was used to elicit the responses of experts to four 
open-ended research questions related to engineering design in technology education. Fifty-nine 
possible participants were identified by a panel of 12 engineering design experts who had been 
recommended by Dr. Clive Dym. Twenty-two (22) of the 59 persons agreed to participate in the 
study although only 15 actually completed the Round One survey instrument. Table 9 displays 
the actual number of those completing each survey for each of the four rounds of the Delphi 
research process. 


Table 9 


Completers by Round 


Round # 

Number 

Round 1 

N=15 

Round 2 

N=13 

Round 3 

N=13 

Round 4 

N=12 


Table 10 displays the demographic data of the participants who completed the Round One survey 
instrument. 
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Table 10 


Demographic Data for Round One Participants: N=15 


Gender 

Average vears 
of experience 

Highest level 
of education 

Area of 
expertise 

Current 

emplovment 

Male n-14 

23.4 

Ph.D. n=13 

Mechanical Engineering 
n=12 

University 

n=14 

Female n=l 


Post-doctoral 

n=2 

Design n=l 

Industry n=I 




Mechanical and Industrial 
Engineering n=l 





Design Methodology n=l 



The study was conducted via the Internet and participants completed and submitted all survey 
instruments electronically. Data from previous completers was retained in subsequent rounds 
even if they did not complete additional surveys (Ludlow, 2002) because this input can be 
considered important and valid even if the participant does not complete subsequent rounds. 

It is important to note that each of the participants completing all rounds in this Delphi 
research process had a background in mechanical engineering. They were also all employed in 
academic settings except for one. This commonality among participants provides strength and 
focus for the study in that it is easy to categorize the results of this study and compare them to 
the results of other studies with similarly homogenous groups. Based on personal correspondence 
with Dr. David Gattie at UGA (November 11, 2006), this mechanical engineering focus does 
provide some possible areas of bias, especially in the area of prototyping and testing. 

Round 1 

The Round 1 survey instrument was made available to participants online from June 19 to 
June 28, 2006. Each participant was contacted via email and directed to access the study website 
in order to record their responses to the four research questions: 

1 . What aspects of the engineering design process best equip secondary students to 
understand, manage, and solve technological problems? 
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2. What mathematics concepts related to engineering design should secondary students 
use to understand, manage, and solve technological problems? 

3. What specific science principles related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

4. What specific skills, techniques, and engineering tools related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 

The survey instrument was completed by 15 of the 22 persons who had agreed to participate. 
Establishing Validity 

A total of 234 responses (see Appendix D) were received from the 15 participants 
during Round 1 . In order to establish content validity, this data was sent to Dr. Paul Schrueders 
and Dr. Tim Taylor, engineering professors at Utah State University, so that they could review 
the entire list of responses and condense the data into a list of unique items. The professional 
literature regarding the Delphi research process recommends a panel of at least two persons to 
monitor this process (Turoff, 1970) of identifying the items that will form the Round 2 survey 
instrument. Table 1 1 contains the reviewed list of all unique responses as created by Drs. 
Schrueders and Taylor. 


Table 11 

Results of Round One Data Review 
Question 1 

What aspects of the engineering design process best equip secondary students to understand, 
manage, and solve technological problems 

1 . Understand 

1.1. Problem identification/formulation/development of requirements lists 

1.2. Functional structures 

1.3. Customer needs 

2. Manage 

2.1. Project planning and scheduling 

2.2. Teamwork 
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2.3. Decision making methodologies 

2.4. Communication 

2.4.1. Written 

2.4.2. Oral 

2.4.3. Graphical/pictorial 

2.5. Negotiation 

2.6. Meeting skills 

2.7. Personal ethics 

2.8. Multicultural/diversity awareness 
3. Solve 

3.1. Ability to break down complex problems in manageable pieces 

3.2. Ability to handle open-ended/ill-defined problems 

3.3. Ability to integrate multiple domains of knowledge 

3.4. Acceptance of multiple solutions to a single problem 

3.5. Brainstorming and innovative concept generation 

3.6. Conceptual design 

3.7. Design for robustness/failure mode analysis 

3.8. Engineering heuristics for/analysis-based design 

3.9. Experimental design, data collection, and interpretation of results 

3.10. Eunctional product modeling 

3.11. Human factors and safety in design 

3.12. Identification of good/bad design 

3.13. Identification of underlying scientific principles 

3.14. Problem identification/ formulation/ development of requirements lists 

3.15. Product optimization 

3.16. Product testing/ functional analysis 

3.17. Prototyping/fabrication skills 

3.18. Recognition that the solution method depends on the type of problem at hand 

3.19. Research/library skills 

3.20. Simplicity and clarity of use and function 

3.21 . Synthesis of simple parts into more complex system. 

3.22. Understanding product life cycles/life cycle analysis 


4. General Skills 

4.1. 

Critical thinking 

4.2. 

Experience 

4.3. 

Logic and logical thinking 

4.4. 

Systems thinking 

Question 2 



What mathematics concepts related to engineering design should secondary students use to 
understand, manage, and solve technological problems? 
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1 . Algebra 

1.1. Basic Algebra 

1.2. Advanced Algebra 

1.3. Linear Algebra 

2. Geometry 

3. Trigonometry 

4. Pre-Calculus 

5. Statistics 

6. Calculus 

6.1. Integration 

6.2. Differentiation 

6.3. Differential equations 

7. Skills and Concepts 

7.1. Measurement theory 

7.2. Approximation 

8. Problem formulation 

8.1. Ability to handle open-ended/ill-defined problems 

8.2. Multiple solutions to a single problem 

8.3. Optimization 

9. Computer Skills 

9.1. Programming 

9.2. Spreadsheets 

9.3. Modeling/simulation/numerical analysis software 
Question 3 

What specific science principles related to engineering design should secondary students use to 
understand, manage, and solve technological problems? 

1 . Chemistry 

1.1. Materials properties 

1 .2. Effects chemical formulation on manufacturing 

2. Biology 

2.1. Evolution 

3. Physics 

3.1. Conservation of mass, energy, and momentum 

3.2. Dynamic systems 

3.3. Introductory mechanics 

3.4. Newton’s laws: forces, reactions, velocity & acceleration 

3.5. Summation of forces/force equilibrium 

3.6. Types of energy 

4. Engineering 

4.1. Circuit analysis and electrical power 

4.2. Control theory 

4.3. Eluidfiow 

4.4. Heat and mass balances 
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4.5. Heat transfer 

4.6. Statics 

4.7. Strength of materials 

4.8. Thermodynamics 

5. Education/Learning Theory 

5.1. Decision analysis 

5.2. Cognitive science 

5.3. Learning theories 

Question 4 

What specific skills, techniques, and engineering tools related to engineering design should 
secondary students use to understand, manage, and solve technological problems? 

1 . Proj ect Management 

1.1. Communications 

1.1.1. Written 

1.1.2. Oral 

1.1.3. Graphical/pictorial 

1 .2. Decision making principles and methods 

1.3. Meeting skills 

1.4. Multicultural/ diversity awareness 

1.5. Negotiation 

1.6. Personal ethics 

1 .7. Project planning and scheduling 

1.8. Teamwork 

2. Computer Skills 

2.1. Computer aided design software 

2.2. Computer searching 

2.3. E-mail 

2.4. Modeling/simulation/numerical analysis software 

2.5. Plotting Software 

2.6. Presentation software 

2.7. Spread Sheets 

3. General Skills 

3.1. Ability to abstract 

3.2. Ability to synthesize 

3.3. Analogical reasoning 

3.4. Common sense 

3.5. Critical thinking 

3.6. Historical perspective 

3.7. Logical thinking 

4. Problem Solving/Design Skills 

4.1. Ability to break down complex problems in manageable pieces. 

4.2. Ability to handle open-ended/ill-defined problems 

4.3. Ability to integrate multiple domains of knowledge 
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4.4. Acceptance of multiple solutions to a single problem 

4.5. Analysis-based design 

4.6. Basic mechanical mechanisms 

4.7. Brainstorming and innovative eoneept generation 

4.8. Conceptual design 

4.9. Dealing with multiple solutions to a single problem 

4.10. Design for robustness/ failure mode analysis 

4.11. Engineering heuristics for/analysis-based design 

4.12. Experimental design, data colleetion, and interpretation of results 

4.13. E ailure mode and effects analysis 

4.14. Eunctional product modeling 

4.15. Human factors and safety in design 

4.16. Identifieation of good/bad design 

4.17. Identifieation of underlying seientific prineiples 

4.18. Problem identifieation/ formulation/ development of requirements lists 

4.19. Product disseetion 

4.20. Product optimization 

4.2 1 . Product testing/functional analysis 

4.22. Prototyping/fabrication skills 

4.23. Recognition that the solution method depends on the type of problem at hand. 

4.24. Researeh/library skills 

4.25. Synthesis of simple parts into more complex system. 

4.26. Understanding produet life cycles/life cyele analysis 


Round 2 

The list of unique responses identified by Drs. Schrueders and Taylor during the review 
proeess (see Table 11) became the items in the Round 2 survey instrument (see Appendix E). The 
online survey was made available from July 10 to July 19, 2006. Participants were eontacted via 
email and direeted to access the online survey in order to indieate their level of agreement with 
eaeh item on a 6-point Eikert scale. Table 12 displays the format for eaeh item on the survey. 
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Table 12 


Example: Item 1 

1 . Understand problem identification / formulation / development of requirements lists 


Strongly 

Disagree 

Disagree 

Somewhat 

Disagree 

Somewhat 

Agree 

Agree 

Strongly 

Agree 

1 

2 

3 

4 

5 

6 


Thirteen of the original 15 participants from Round 1 completed the survey by the July 19 
deadline. Table 13 displays the results. 


Table 13 

Round Two Results 


Research Question One: What aspects of the engineering design process best 
equip secondary students to understand, manage, and solve technological 
problems? 

ITEM 

Mean 

Median 

SD 

IQR 

I . Understand problem 
identification/ formulation 
/development of requirements lists 

5.85 

6 

0.3755 

6 

2. Understand functional structures 

4.77 

5 

1.3634 

4-6 

3. Understanding of customer needs 

5.31 

5 

0.8549 

5-6 

4. Project planning and scheduling 

4.92 

5 

0.7596 

4-5 

5. Teamwork 

5.23 

5 

0.5991 

5-6 

6. Decision making methodologies 

4.62 

5 

1.0439 

4-5 

7. Written communication 

5.23 

5 

0.8321 

5-6 

8. Oral communication 

5.54 

6 

0.5189 

5-6 

9. Graphical/pictorial communication 

5.23 

5 

0.7250 

5-6 

10. Negotiation 

4.15 

4 

0.8006 

4 

1 1 . Meeting skills 

4.54 

4 

0.6602 

4-5 

12. Personal ethics 

5.00 

5 

0.7071 

5 

13. Multicultural/diversity awareness 

4.38 

5 

1.2609 

4-5 

14. Ability to break down complex 
problems in manageable pieces 

5.00 

5 

0.8165 

4-6 
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15. Ability to handle open-ended/ill 
defined problems 

5.46 

6 

0.6602 

5-6 

16. Ability to integrate multiple 
domains of knowledge 

5.31 

6 

0.9473 

5-6 

17. Aeeeptance of multiple 
solutions to a single problem 

5.62 

6 

0.5064 

5-6 

18. Brainstorming and innovative 
coneept generation 

5.00 

5 

0.5774 

5 

19. Conceptual design 

5.42 

5 

0.5149 

5-6 

20. Design for robustness/failure 
mode analysis 

3.85 

4 

1.0682 

4 

21. Engineering heuristics for 
analysis-based design 

3.92 

4 

1.3790 

3.75-5 

22. Experimental design, data 
collection, and interpretation of 
results 

4.46 

4 

0.7763 

4-5 

23. Eunctional product modeling 

4.31 

4 

0.8549 

4-5 

24. Human factors and safety in 
design 

4.54 

5 

0.5189 

4-5 

25. Identification of good/bad design 

4.77 

5 

0.7250 

4-5 

26. Identification of underlying 
scientific principles 

4.77 

5 

0.8321 

4-5 

27. Product optimization 

3.54 

4 

0.8771 

3-4 

28. Product testing/functional analysis 

4.46 

5 

0.8771 

4-5 

29. Prototyping/fabrication skills 

4.85 

5 

0.8006 

4-5 

30. Recognition that the solution 
method depends on the type of 
problem at hand 

4.31 

4 

0.8549 

4-5 

3 1 . Research/library skills 

4.62 

5 

1.0439 

4-5 

32. Simplicity and clarity of use 
and function 

4.54 

4 

0.6602 

4-5 

33. Synthesis of simple parts into 
more complex system 

4.54 

5 

0.7763 

4-5 

34. Understanding product life 
cycles/life cycle analysis 

4.38 

4 

0.7679 

4-5 

35. Critical thinking 

5.23 

5 

0.8321 

5-6 

36. Experience 

4.00 

4 

1.2910 

3-5 

37. Logic and logical thinking 

4.92 

5 

0.6405 

5 

38. Systems thinking 

5.31 

6 

0.8549 

5-6 
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Research Question Two: What mathematics concepts related to engineering 
design should secondary students use to understand, manage, and solve 
technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

40. Basic Algebra 

5.38 

6 

0.7679 

5-6 

4 1 . Advanced Algebra 

4.85 

5 

1.1435 

4-6 

42. Linear Algebra 

4.00 

4 

1.0000 

3-5 

43. Geometry 

5.15 

5 

0.6887 

5-6 

44. Trigonometry 

5.17 

5 

0.7177 

5-6 

45. Pre-Calculus 

4.75 

5 

0.6216 

4-5 

46. Statistics 

4.85 

5 

0.8987 

4-6 

47. Calculus- Integration 

4.62 

5 

0.9608 

4-5 

48. Calculus- Differentiation 

4.62 

5 

0.9608 

4-5 

49. Calculus- Differential Equations 

3.92 

4 

0.7596 

4 

50. Measurement theory 

4.23 

4 

0.7250 

4 

5 1 . Approximation 

5.08 

5 

1.0377 

4-6 

52. Ability to handle open-ended/ill 
defined problems 

5.62 

6 

0.6504 

5-6 

53. Multiple solutions to a 
single problem 

5.46 

6 

0.6602 

5-6 

54. Optimization 

3.31 

4 

1.1094 

3-4 

55. Computer Programming 

4.23 

4 

1.0919 

4-5 

56. Spreadsheets 

5.15 

5 

0.8006 

5-6 

57. Modeling/simulation/numerical 
analysis software 

4.46 

4 

0.9674 

4-5 

Research Question Three: What specific science principles related to engineering 
design should secondary students use to understand, manage, and solve 
technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

59. Chemical properties of materials 

4.17 

4 

0.5774 

4-4.25 

60. Effects of chemical formulation 
on manufacturing 

3.38 

3 

1.0439 

3-4 

61. Biological Evolution 

3.31 

3 

1.3775 

2-4 

62. Conservation of mass, energy, 
and momentum 

- 


- 

- 

63. Dynamic systems 

4.23 

4 

0.5991 

4-5 

64. Introductory mechanics 

4.77 

5 

0.7250 

4-5 

65. Newton’s laws: forces, reactions, 
velocity & acceleration 

5.31 

5 

0.6304 

5-6 
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66. Summation of forces/force 
equilibrium 

5.08 

5 

0.6405 

5 

67. Types of energy 

5.23 

5 

0.5991 

5-6 

68. Circuit analysis and electrical 
power 

4.08 

4 

0.4935 

4 

69. Control theory 

3.00 

3 

1.0000 

2-4 

70. Fluid flow 

3.77 

4 

0.5991 

3-4 

71. Heat and mass balances 

3.92 

4 

0.7596 

3-4 

72. Heat transfer 

4.08 

4 

0.7930 

4 

73. Statics 

5.00 

5 

0.7385 

4-75-5.25 

74. Strength of materials 

4.69 

5 

0.9473 

4-5 

75. Thermodynamics 

3.85 

4 

1.0682 

3-4 

76. Decision analysis 

3.77 

3 

1.4233 

3-4 

77. Cognitive science 

3.62 

4 

1.3868 

2-5 

78. Learning theories 

3.62 

3 

1.5021 

3-4 

Research Question Four: What speciflc skills, techniques, and engineering tools 

related to engineering design should secondary students use to understand, 
manage, and solve technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

80. Computer aided design software 

4.46 

5 

0.8771 

4-5 

8 1 . Computer searching 

4.38 

5 

1.3253 

4-5 

82. E-mail 

4.83 

5 

0.8348 

4.75-5 

83. Plotting Software 

4.62 

5 

0.7679 

4-5 

84. Presentation software 

4.85 

5 

0.8987 

4-5 

85. Ability to abstract 

5.23 

5 

0.8321 

5-6 

86. Ability to synthesize 

5.69 

6 

0.4804 

5-6 

87. Analogical reasoning 

5.08 

5 

0.6686 

5-5.25 

88. Historical perspective 

4.46 

4 

0.8771 

4-5 

89. Analysis-based design 

4.23 

4 

0.9268 

4 

90. Basic mechanical mechanisms 

4.23 

4 

0.7250 

4-5 

91. Eailure mode and effects analysis 

3.85 

4 

0.9871 

3-4 

92. Product Dissection 

5.23 

5 

0.5991 

5-6 


The Round 2 survey also included space for participants to add additional items they felt 
should be included in order to more fully answer the four research questions. Table 14 displays 
the comments and new survey items submitted by participants during Round 2. 
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Table 14 


Round 2 Additions and Comments 

Research Question 1 

- The above is quite a comprehensive list. 

- Some basics of costing, profit, and economic analysis 

- Understanding the context within which the technological problem exists and the possible 
effects of external infiuences on the situation. 

- You listed most topics in my book “The Mechanical Design Process” so I naturally think they 
are all very important or close to it. You got most of them. 

- Product architecture and modularity/interfaces; design principles to assist in generating 
innovative concepts; design-by-analogy and analogical reasoning to generate innovative 
concepts 

- Nearly all the above are important— the issue is deciding which ones are most important. 

Research Question 2 

- Algebraic equations for gear ratios, conservation of energy, conservation of momentum, 
projectile motion, structural equilibrium, basic stresses 

- I think this is all backwards the attraction of designing and making their own stuff can 
motivate the students to learn math. I wouldn’t focus on what they need to know, but focus on 
what they can learn by being motivated to create things. 

Research Question 3 

- This depends a great deal on the types of engineering design problems presented 

- Project management 

- Thermal expansion and contraction 

Research Question 4 

- Reverse engineering 

- Finishing a job to the last detail 

- Recognize team roles and personality types 


Round 3 

The Round 3 survey instrument (see Appendix G) was made available online from July 
24, 2006 to August 2, 2006. Each participant was emailed Round 2 survey responses to remind 
each of the previous choices. The 13 participants who completed Round 2 also completed 
this survey by the deadline. The survey contained all survey items from Round 2 along with 
statistical data. The mean, maximum, minimum, standard deviation and interquartile range were 
calculated for each item and displayed for the participants. Table 15 displays item one from the 
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survey as an example of how each item appeared to participants. At this point a numeric score 
was also displayed alongside each choice so that the statistical data would be readily understood 
by participants. 


Table 15 

Item 1 - Round 3 (Round 2 statistical data included). 

1 . Understand problem identification / formulation / development of requirements lists (Round 
Two Data: Mean= 5.85, Max=6, Min=5, St. Dev.= 0.3755, IQR= 6) 


Strongly 

Disagree 

Disagree 

Somewhat 

Disagree 

Somewhat 

Agree 

Agree 

Strongly 

Agree 

1 

2 

3 

4 

5 

6 


In addition to the original items and corresponding statistical data, fifteen new items 
suggested by participants in Round 2 were added to the Round 3 survey instrument. Since these 
were new items, they were identified as such and had no statistical data brought forward from the 
previous round. Table 16 displays the results of Round 3. 
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Table 16 


Round 3 Data 


Research Question 1 : What aspects of the engineering design process best 
equip secondary students to understand, manage, and solve technological 
problems? 

ITEM 

Mean 

Median 

SD 

IQR 

1 . Understand problem 
identification/ formulation 
/development of 
requirements lists 

5.38 

6 

1.3868 

5-6 

2. Understand functional 
structures 

4.00 

4 

1.5811 

4-5 

3. Understanding of customer 
needs 

5.00 

5 

1.4142 

5-6 

4. Project planning and 
scheduling 

4.54 

4 

0.8771 

4-5 

5. Teamwork 

5.31 

5 

0.6304 

5-6 

6. Decision making 
methodologies 

4.58 

4.5 

1.0836 

4-5.25 

7. Written communication 

5.46 

6 

0.6602 

5-6 

8. Oral communication 

5.54 

6 

0.5189 

5-6 

9. Graphical/pictorial 
communication 

5.54 

6 

0.5189 

5-6 

10. Negotiation 

4.46 

4 

0.9674 

4-5 

1 1 . Meeting skills 

4.62 

5 

0.6504 

4-5 

12. Personal ethics 

5.15 

5 

0.6887 

5-6 

1 3 . Multicultural/ diversity 
awareness 

4.08 

4 

1.1152 

4 

14. Ability to break down 
complex problems in 
manageable pieces 

5.17 

5 

0.7177 

5-6 

15. Ability to handle open- 
ended/ill defined problems 

5.77 

6 

0.4385 

6 

16. Ability to integrate 
multiple domains of 
knowledge 

5.08 

5 

1.1152 

5-6 
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17. Acceptance of multiple 
solutions to a single problem 

5.77 

6 

0.4385 

6 

18. Brainstorming and 
innovative eoneept 
generation 

5.15 

5 

0.8006 

5-6 

19. Coneeptual design 

5.23 

5 

0.7250 

5-6 

20. Design for robustness/ 
failure mode analysis 

3.54 

4 

1.2659 

3-4 

2 1 . Engineering heuristies for 
analysis-based design 

3.75 

4 

1.3568 

3-4.25 

22. Experimental design, data 
eolleetion, and interpretation 
of results 

4.54 

5 

0.5189 

4-5 

23. Eunetional product 
modeling 

4.46 

5 

0.8771 

4-5 

24. Human factors and safety 
in design 

4.62 

5 

0.5064 

4-5 

25. Identifieation of good/bad 
design 

4.62 

5 

0.7679 

4-5 

26. Identifieation of 
underlying seientifie 
prineiples 

4.92 

5 

0.8623 

4-6 

27. Product optimization 

3.00 

3 

0.8165 

3 

28. Product testing/functional 
analysis 

4.38 

4 

0.7679 

4-5 

29. Prototyping/fabrication 
skills 

4.77 

5 

0.5991 

4-5 

30. Recognition that the 
solution method depends on 
the type of problem at hand 

4.62 

4 

0.7679 

4-5 

3 1 . Researeh/library skills 

4.85 

5 

0.8006 

4-5 

32. Simplicity and clarity of 
use and funetion 

4.77 

5 

0.8321 

4-5 

33. Synthesis of simple parts 
into more complex system 

4.69 

5 

0.6304 

4-5 

34. Understanding produet 
life eycles/life eycle analysis 

4.38 

4 

0.8697 

4-5 

35. Critieal thinking 

5.23 

5 

0.8321 

5-6 

36. Experience 

3.62 

3 

1.1929 

3-4 

37. Logie and logieal 
thinking 

4.85 

5 

0.6887 

4-5 

38. Systems thinking 

5.69 

6 

0.4804 

5-6 

39a. Costing, profit, and basie 
eeonomie analysis 

3.92 

4 

0.8623 

4 
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39b. Understanding the 
context of the technological 
problem and possible 
external influences 

4.77 

5 

1.1658 

5 

39c. Product architecture and 
modularity/interfaces 

3.92 

4 

1.1875 

3-4 

39d. Design principles 
to assist in generating 
innovative concepts 

4.38 

4 

0.9608 

4-5 

39e. Design by analogy 

4.00 

4 

0.8165 

4 

Research Question 2: What mathematics concepts related to engineering 
design should secondary students use to understand, manage, and solve 
technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

40. Basic Algebra 

5.54 

6 

0.6602 

5-6 

4 1 . Advanced Algebra 

4.62 

5 

1.1929 

4-5 

42. Linear Algebra 

3.62 

4 

1.1209 

3-4 

43. Geometry 

5.46 

6 

0.7763 

5-6 

44. Trigonometry 

5.00 

5 

0.9129 

5-6 

45. Pre-Calculus 

4.62 

4 

1.1209 

4-5 

46. Statistics 

4.23 

5 

1.5892 

3-5 

47. Calculus- Integration 

3.77 

4 

1.7394 

2-5 

48. Calculus- Differentiation 

3.77 

4 

1.7394 

2-5 

49. Calculus- Differential 
Equations 

3.08 

4 

1.3205 

2-4 

50. Measurement theory 

3.46 

4 

1.4500 

2-4 

5 1 . Approximation 

4.54 

5 

1.0500 

4-5 

52. Ability to handle open- 
ended/ill defined problems 

5.54 

6 

0.6602 

5-6 

53. Multiple solutions to a 
single problem 

5.69 

6 

0.4804 

5-6 

54. Optimization 

2.85 

3 

1.1435 

2-3 

55. Computer Programming 

3.92 

4 

1.3821 

4 

56. Spreadsheets 

5.23 

5 

0.9268 

5-6 

57. Modeling/simulation/ 
numerical analysis software 

3.92 

4 

1.5525 

3-5 

58a. Algebraic equations for 
determining gear ratios 

2.75 

3 

1.4222 

1.75-3 

58b. Conservation of 
momentum 

4.25 

4 

1.2154 

4-5 

58c. Projectile motion 

3.45 

4 

1.4397 

3-4 

58d. Structural equilibrium 

4.33 

4 

0.9847 

4-5 

58e. Basic stresses 

4.25 

4 

1.2154 

3.75-5 
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Research Question 3; What specific science principles related to engineering 
design should secondary students use to understand, manage, and solve 
technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

59. Chemical properties of 
materials 

3.83 

4 

1.0299 

4 

60. Effects of chemical 
formulation on 
manufacturing 

2.75 

3 

0.7538 

2.75-3 

61. Biological Evolution 

3.17 

3 

1.5859 

2.5-4 

62. Conservation of mass, 
energy, and momentum 

4.92 

5 

1.2401 

4-6 

63. Dynamic systems 

4.08 

4 

1.0836 

4-5 

64. Introductory mechanics 

4.45 

4 

0.9342 

4-5 

65. Newton’s laws: forces, 
reactions, velocity & 
acceleration 

5.42 

5.5 

0.6686 

5-6 

66. Summation of forces/ 
force equilibrium 

5.00 

5 

0.6030 

5 

67. Types of energy 

5.25 

5 

0.6216 

5-6 

68. Circuit analysis and 
electrical power 

3.75 

4 

0.6216 

4 

69. Control theory 

2.83 

3 

1.3371 

2.5-3 

70. Eluid flow 

3.42 

4 

0.7930 

3-4 

71. Heat and mass balances 

3.58 

4 

0.7930 

3-4 

72. Heat transfer 

3.58 

4 

0.7930 

3-4 

73. Statics 

4.50 

5 

1.0000 

4-5 

74. Strength of materials 

4.25 

4.5 

1.1382 

3.75-5 

75. Thermodynamics 

3.25 

3 

0.8660 

3-4 

76. Decision analysis 

3.33 

3 

1.1547 

3-4 

77. Cognitive science 

3.08 

3 

1.1645 

2-4 

78. Learning theories 

3.58 

3 

1.6214 

3-4.5 

79a. Project management 

4.17 

4 

1.2673 

3-5 

79b. Thermal expansion/ 
contraction 

3.92 

4 

0.7930 

4 
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Research Question 4: What specific skills, techniques, and engineering tools 
related to engineering design should secondary students use to understand, 
manage, and solve technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

80. Computer aided design 
software 

4.33 

4.5 

1.2309 

3.75-5 

8 1 . Computer searching 

4.92 

5 

0.7930 

4-5.25 

82. E-mail 

5.18 

5 

0.6030 

5-5.5 

83. Plotting Software 

4.50 

4.5 

0.7977 

4-5 

84. Presentation software 

5.00 

5 

0.7385 

4.75-5.25 

85. Ability to abstract 

5.17 

5 

0.7177 

5-6 

86. Ability to synthesize 

5.75 

6 

0.4523 

5.75-6 

87. Analogical reasoning 

5.17 

5 

0.7177 

5-6 

88. Historical perspective 

4.42 

4.5 

1.1645 

4-5 

89. Analysis-based design 

4.25 

4 

1.4222 

4-5.25 

90. Basic mechanical 
mechanisms 

4.17 

4 

0.8348 

4-4.25 

91. Eailure mode and effects 
analysis 

3.00 

3 

1.2060 

2.75-4 

92. Product Dissection 

4.58 

5 

0.9962 

4-5 

93 a. Reverse engineering 

4.17 

4 

1.4668 

4-5 

93b. Einishing job to the last 
detail 

3.92 

4 

1.4434 

3-5 

93 c. Recognizing team roles 
and personality types 

4.58 

4 

0.7930 

4-5 


As in Round 2, participants had the opportunity to add any additional items they felt 
would help to answer the four research questions. Eight additional items were suggested by 
participants and these items were added to the Round 4 survey instrument. In addition to having 
the opportunity to add new survey items, participants were encouraged to provide an explanation 
of their answer on any particular item. Table 17 contains both the new items suggested and the 
comments given by participants whose ehoiees were outside the interquartile range. 


62 


Table 17 


Round 3 Additions and Comments 

Question 1 

- More advanced high school students may be interested in understanding basic business 
motivations for engineering design, such as marketing or consumer research 

- These questions have done a good job covering the topics. Perhaps basic manufacturing 
processes could be another topic. 

- House of Quality methodology 

Question 2 

- The following are getting too fine! ! 

- Scaling. Using geometry & trig to understand how to scale the size of a component up or 
down. 

- Any formula expressing the performance of a system (there is no one single formula since 
each system is unique). 

- No - 1 think that students should be getting turned on to making and designing new things - 
and understanding what engineers do. The design should motivate the math learning, not the 
other way around. 

Question 3 

- leadership principles, environmentally conscious principles 

- Question Asking (Inquiry) 

- Again, I think that engineering for high school students should not turn into mini science 
courses. They should focus on what is different about engineering: creating new stuff 
Whatever math/science is needed for the particular problem should be taught/learned. The 
program shouldn’t be driven by delivering particular math/science content. 

Question 4 

- Engineering intuition. It is important in all this that students don’t get caught up in entirely in 
the calculations. Several researchers have discussed the need to develop a student’s sense of a 
problem first (Margot Brereton calls it “Synalysis”). 

Qutside IQR Explanation 

- Eor Q61, 1 put “1” since biological evolution has some scientific evidence to support it at the 
micro-evolution level but no evidence to support it at the macro-level. The concept of micro- 
evolution would be somewhat valuable to learn, but I don’t feel like the general concept of 
evolution would be valuable. 

- 7. We now that informal communication plays an under-recognized role in design team 
interactions. Although it is highly relevant, formal communication is overemphasized in 
design/engineering education— maybe because it is easier to teach formal communication 
practices than informal communication practices. 10. There is a large body of evidence 
that suggests design, among many other things, IS negotiation. Negotiation requires mental 
fiexibility, which is critical for being able to understand and reframe context. 13. If one is to 
understand and relate to users from diverse backgrounds, and drive his/her design process with 
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that understanding, one must have the highest awareness of multicultural issues. 69. Control 
theory is applicable to all engineering domains and is critical for understanding the behavior 
of all engineering systems. 76. Decision analysis is applicable to design practice in all 
engineering domains, and facilitates the application of well-considered design processes. 78. 
Design is a learning-intensive activity. One cannot master designing without understanding 
his/her learning processes. 


Round 4 

The Round 4 survey was available online from August 7 to 16, 2006. Since the literature 
supports a three-round Delphi (Tinstone & Murray, 1975) and also indicates that most changes 
will occur in early rounds of the Delphi study (Dalkey & Helmer, 1963; Dalkey, 1968), it was 
decided to only include items in the Round 4 survey instrument (see Appendix I) that met one or 
more of the following criteria: 

1 . Items that had a mean shift of >15% between Round 2 and Round 3 was considered to 
be unstable and were included in Round 4. 

2. Items with an Interquartile Range of >1 had not reached the level of consensus desired 
and was included in Round 4. 

3. Items which had been commented on during Round 3 were included in Round 4 along 
with the comments so that all participants could see their colleagues’ feedback. 

4. Items that were added in Round 3 were included in Round 4. 

Fifty items fell into one or more of these categories and were included in the Round 4 
survey instrument (see Appendix I). Each participant was emailed Round 3 survey responses 
to remind each of the previous choices. Twelve of the thirteen participants who completed the 
Round 3 survey accessed and completed the Round 4 survey by the deadline. Each item on the 
survey that was brought forward from previous rounds had the associated statistical data (mean, 
maximum, minimum, standard deviation, and interquartile range) listed beside the question. 

In addition, any comments made by participants whose previous answers were outside the 
interquartile range (IQR) were also listed along with the survey item. Table 1 8 displays the 
results of the Round 4 survey. 
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Table 18 


Round 4 Results 


Research Question 1 : What aspects of the engineering design process best equip 
secondary students to understand, manage, and solve technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

2. Understand functional structures 

4.25 

4 

1.5448 

4-5.25 

6. Decision making methodologies 

4.58 

5 

1.1645 

4-5 

7. Written communication 

5.08 

5 

0.9003 

5-6 

10. Negotiation 

4.42 

4 

0.7930 

5-6 

13. Multicultural/diversity awareness 

4.08 

4 

1.0836 

4 

21. Engineering heuristics for 
analysis-based design 

3.45 

4 

1.4397 

2.5-4 

26. Identification of underlying 
scientific principles 

5.08 

5 

0.7930 

4.75-6 

27. Product optimization 

3.08 

3 

0.9962 

3-3.25 

39a. Costing, profit, and basic economic 
analysis 

3.50 

3.5 

1.000 

3-4 

39b. Understanding the context of the 
technological problem and possible 
external infiuences 

4.75 

5 

0.9653 

4-5.25 

39c. Product architecture and modularity/ 
interfaces 

3.67 

4 

1.1547 

3-4 

39d. Design principles to assist in 
generating innovative concepts 

4.67 

5 

0.8876 

4-5 

39e. Design by analogy 

4.17 

4 

0.5774 

4 

39f Understanding basic business 
motivations for engineering design, such as 
marketing or consumer research 

4.00 

4 

1.1282 

3.75-5 

39g Understanding basic manufacturing 
processes 

4.25 

4 

0.8660 

4-5 

39h House of Quality method 

3.25 

3 

1.3568 

2.75 

Research Question 2: What mathematics concepts related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 

ITEM 

Mean 

Median 

SD 

IQR 

46. Statistics 

4.25 

4 

1.3568 

4-5 

47. Calculus- Integration 

3.25 

3.5 

1.2881 

2-4 

48. Calculus- Differentiation 

3.17 

3.5 

1.4035 

2-4 
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49. Calculus- Differential Equations 

2.73 

3 

1.2721 

2-3.5 

50. Measurement theory 

3.17 

3.5 

1.2673 

2.75-4 

57. Modeling/simulation/numerical 
analysis software 

3.64 

3 

1.6895 

3-5 

58a. Algebraie equations for determining 
gear ratios 

2.83 

2.5 

1.5275 

3-5 

58b. Conservation of momentum 

4.08 

4 

1.2401 

4-5 

58c. Projectile motion 

3.25 

3 

1.3568 

3-4 

58d. Struetural equilibrium 

4.50 

4 

0.6742 

4-5 

58e. Basic stresses 

4.42 

4 

0.7930 

4-5 

58f. Using geometry and trigonometry to 
change the scale of a component 

3.92 

4 

0.7930 

3-4.25 

58g. Formulas capable of expressing the 
performance of a system 

4.25 

4 

1.2154 

3.75-5 

Research Question 3: What specific science principles related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 

ITEM 

Mean 

Median 

SD 

IQR 

60. Effects of chemical formulation 
on manufacturing 

2.50 

2.5 

0.7977 

2-3 

61. Biological Evolution 

3.00 

3 

1.4142 

2-4 

62. Conservation of mass, energy, 
and momentum 

4.67 

4.5 

0.9847 

4-5.25 

69. Control theory 

2.83 

3 

1.3371 

1.75-4 

74. Strength of materials 

4.42 

5 

1.0836 

4-5 

75. Thermodynamics 

3.33 

3 

0.8876 

3-4 

76. Decision analysis 

3.67 

3.5 

1.4355 

3-5 

78. Learning theories 

3.50 

3 

1.5076 

3-4.25 

79a. Project management 

4.25 

4 

1.0553 

3.75-5 

79b. Thermal expansion/contraction 

4.00 

4 

0.8528 

3.75-4 

79c. Question asking -inquiry 

4.58 

5 

0.7930 

4-5 

79d. Leadership principles 

3.58 

4 

1.0836 

3.75-4 

79e. Principles related to environmental 
consciousness 

4.42 

4.5 

0.6686 

4-5 
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Research Question 4: What specific skills, techniques, and engineering tools related 
to engineering design should secondary students use to understand, manage, and 
solve technological problems? 

ITEM 

Mean 

Median 

SD 

IQR 

80. Computer aided design software 

4.00 

4 

1.1282 

3.75-5 

8 1 . Computer searching 

4.67 

5 

0.8876 

4-5 

89. Analysis-based design 

4.17 

4 

1.2673 

4-5 

91. Failure mode and effects analysis 

3.08 

3 

1.2401 

2.75-4 

93 a. Reverse engineering 

4.17 

4 

1.1934 

4-5 

93b. Finishing job to the last detail 

3.67 

3.5 

1.4975 

3-4.25 

93 c. Recognizing team roles and 
personality types 

4.58 

4.5 

0.6686 

4-5 

93 d. Engineering intuition 

3.45 

4 

1.3685 

2.5-4.5 


Final Results 

The final results for each item appear below in Table 19. In addition to the mean, median, 
standard deviation, and interquartile range scores, the mean shift during the previous two rounds 
is reported for each item. This score indicates the degree of stability for each individual item, 
while the IQR indicates the level of consensus afforded the item by the participants. As described 
in the methods section of this study, an IQR score of < I is considered to be an indication that the 
item has reached an acceptable degree of consensus. A mean shift (or A Mean) of < 15% is an 
indication that the item can be considered stable. 

The literature was vague as to the method used in attributing different levels of 
significance to the statistical scores that result from Delphi studies. Based upon personal 
correspondence with Wicklein (September 25, 2006) and Rojewski (September 30, 2006), 
a decision was made to maintain the highest standards for the purpose of this study. It was 
determined that applying the most stringent criteria to the data resulting from the Delphi process 
would ensure that only items that were undeniably very important would be placed in the highest 
category and considered in the conclusions and recommendations. All other items would fall 
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into a secondary category of lesser importance. Items considered to be very important for the 
purposes of this research met each of the following criteria: 

1 . An inter-round mean A of <1 5% (indicating stability) 

2. A median score of 5 or 6 (indicating a strong level of agreement among participants) 

3. An IQR range of < 1 (indicating consensus) 

Forty-eight (48) items met these strict requirements and are identified in Table 19 with double 
asterisk (**) symbols. Only these items that met the strictest requirements would be considered 
valid for identifying the essential aspects and related academic concepts of an engineering design 
process in secondary technology education curriculum. 


Table 19 

Final Results by Item 


Research Question I : What aspects of the engineering design process best equip 
secondary students to understand, manage, and solve technological problems? 

ITEM 

Mean 

A 

Mean 

(%) 

Median 

SD 

IQR 

* * I . Understand problem 
identification/ formulation 
/development of requirements lists 

5.38 

-7.97 

6 

1.3868 

5-6 

2. Understand functional structures 

4.25 

-4.35 

4 

1.5448 

4- 

5.25 

**3. Understanding of customer 
needs 

5.00 

-5.80 

5 

1.4142 

5-6 

4. Project planning and scheduling 

4.54 

-7.78 

4 

0.8774 

4-5 

**5. Teamwork 

5.31 

1.51 

5 

0.6304 

5-6 

**6. Decision making methodologies 

4.58 

-2.82 

5 

1.1645 

4-5 

**7. Written communication 

5.08 

4.38 

5 

0.9003 

5-6 

**8. Oral communication 

5.54 

0.03 

6 

0.5189 

5-6 

**9. Graphical/pictorial 
communication 

5.54 

5.91 

6 

0.5189 

5-6 

10. Negotiation 

4.42 

-0.90 

4 

0.7930 

5-6 

**11. Meeting skills 

4.62 

1.80 

5 

0.6504 

4-5 

**12. Personal ethics 

5.15 

3.00 

5 

0.6887 

5-6 

13. Multicultural/diversity awareness 

4.08 

0.00 

4 

1.0836 

4 
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**14. Ability to break down complex 
problems in manageable pieces 

5.17 

3.40 

5 

0.7177 

5-6 

**15. Ability to handle open-ended/ 
ill defined problems 

5.77 

5.65 

6 

0.4385 

6 

**16. Ability to integrate multiple 
domains of knowledge 

5.08 

-4.29 

5 

1.1152 

5-6 

**17. Acceptance of multiple 
solutions to a single problem 

5.77 

2.75 

6 

0.4385 

6 

**18. Brainstorming and innovative 
concept generation 

5.15 

3.00 

5 

0.8006 

5-6 

**19. Conceptual design 

5.23 

-3.45 

5 

0.7250 

5-6 

20. Design for robustness/failure 
mode analysis 

3.54 

-7.96 

4 

1.2659 

3-4 

2 1 . Engineering heuristics for 
analysis-based design 

3.45 

-8.70 

4 

1.4397 

2.5-4 

**22. Experimental design, data 
collection, and interpretation of 
results 

4.54 

1.76 

5 

0.5189 

4-5 

**23. Eunctional product modeling 

4.46 

3.54 

5 

0.8771 

4-5 

**24. Human factors and safety in 
design 

4.62 

1.80 

5 

0.5064 

4-5 

**25. Identification of good/bad 
design 

4.62 

-3.13 

5 

0.7679 

4-5 

26. Identification of underlying 
scientific principles 

5.08 

3.16 

5 

0.7930 

4.75- 

6 

27. Product optimization 

3.08 

2.60 

3 

0.9962 

3- 

3.25 

28. Product testing/functional 
analysis 

4.38 

-1.83 

4 

0.7679 

4-5 

**29. Prototyping/fabrication skills 

4.77 

-1.57 

5 

0.5991 

4-5 

30. Recognition that the solution 
method depends on the type of 
problem at hand 

4.62 

7.25 

4 

0.7679 

4-5 

**31. Research/library skills 

4.85 

5.08 

5 

0.8006 

4-5 

**32. Simplicity and clarity of use 
and function 

4.77 

5.10 

5 

0.8321 

4-5 

**33. Synthesis of simple parts into 
more complex system 

4.69 

3.34 

5 

0.6304 

4-5 

34. Understanding product life 
cycles/life cycle analysis 

4.38 

-0.11 

4 

0.8697 

4-5 

**35. Critical thinking 

5.23 

-0.01 

5 

0.8321 

5-6 

36. Experience 

3.62 

-9.50 

3 

1.1929 

3-4 

**37. Logic and logical thinking 

4.85 

-1.48 

5 

0.6887 

4-5 

**38. Systems thinking 

5.69 

7.20 

6 

0.4804 

5-6 

39a. Costing, profit, and basic 
economic analysis 

3.50 

-12.00 

3.5 

1.000 

3-4 
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39b. Understanding the context 
of the technological problem and 
possible external influences 

4.75 

-0.42 

5 

0.9653 

4- 

5.25 

39c. Product architecture and 
modularity/interfaces 

3.67 

-6.81 

4 

1.1547 

3-4 

**39d. Design principles to assist in 
generating innovative concepts 

4.67 

6.21 

5 

0.8876 

4-5 

39e. Design by analogy 

4.17 

4.08 

4 

0.5774 

4 

39f. Understanding basic business 
motivations for engineering design, 
such as marketing or consumer 
research 

4.00 

N/A 

4 

1.1282 

3.75- 

5 

39g Understanding basic 
manufacturing processes 

4.25 

N/A 

4 

0.8660 

4-5 

39h House of Quality method 

3.25 

N/A 

3 

1.3568 

2.75- 

4 

Research Question 2; What mathematics concepts related to engineering design should 
secondary students use to understand, manage, and solve technological problems? 

ITEM 

Mean 

A 

Mean 

(%) 

Median 

SD 

IQR 

**40. Basic Algebra 

5.54 

2.89 

6 

0.6602 

5-6 

**41. Advanced Algebra 

4.62 

-4.67 

5 

1.1929 

4-5 

42. Linear Algebra 

3.62 

-9.50 

4 

1.1209 

3-4 

**43. Geometry 

5.46 

5.94 

6 

0.7763 

5-6 

**44. Trigonometry 

5.00 

-3.23 

5 

0.9129 

5-6 

**45. Pre-Calculus 

4.62 

-2.74 

5 

1.1209 

4-5 

46. Statistics 

4.25 

0.47 

4 

1.3568 

4-5 

47. Calculus- Integration 

3.25 

-16.00 

3.5 

1.2881 

2-4 

48. Calculus- Differentiation 

3.17 

-18.93 

3.5 

1.4035 

2-4 

49. Calculus- Differential Equations 

2.73 

-12.82 

3 

1.2721 

2-3.5 

50. Measurement theory 

3.17 

6.74 

3 

1.2673 

2.75-4 

**51. Approximation 

4.54 

-10.58 

5 

1.0500 

4-5 

**52. Ability to handle open-ended/ 
ill defined problems 

5.54 

-1.34 

6 

0.6602 

5-6 

**53. Multiple solutions to a 
single problem 

5.69 

4.18 

6 

0.4804 

5-6 

54. Optimization 

2.85 

-13.84 

3 

1.1435 

2-3 

55. Computer Programming 

3.92 

-7.35 

4 

1.3821 

4 

**56. Spreadsheets 

5.23 

1.48 

5 

0.9268 

5-6 

57. Modeling/simulation/numerical 
analysis software 

3.64 

-7.69 

3 

1.6895 

3-5 
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58a. Algebraic equations for 
determining gear ratios 

2.83 

2.83 

2.5 

1.5275 

3-5 

58b. Conservation of momentum 

4.08 

-4.17 

4 

1.2401 

4-5 

58c. Projectile motion 

3.25 

-6.15 

3 

1.3568 

3-4 

58d. Structural equilibrium 

4.50 

3.78 

4 

0.6742 

4-5 

58e. Basic stresses 

4.42 

3.85 

4 

0.7930 

4-5 

58f Using geometry and 
trigonometry to change the scale of a 
component 

3.92 

N/A 

4 

0.7930 

3-4.25 

58g. Formulas capable of expressing 
the performance of a system 

4.25 

N/A 

4 

1.2154 

3.75-5 

Research Question 3: What specific science principles related to engineering design 
should secondary students use to understand, manage, and solve technological problems? 

ITEM 

Mean 

A 

Mean 

(%) 

Median 

SD 

IQR 

59. Chemical properties of materials 

3.83 

-8.08 

4 

1.0299 

4 

60. Effects of chemical formulation 
on manufacturing 

2.50 

-10.00 

2.5 

0.7977 

2-3 

61. Biological Evolution 

3.00 

-5.67 

3 

1.4142 

2-4 

62. Conservation of mass, energy, 
and momentum 

4.67 

-5.35 

4.5 

0.9847 

4-5.25 

63. Dynamic systems 

4.08 

-3.56 

4 

1.0836 

4-5 

64. Introductory mechanics 

4.45 

-6.69 

4 

0.9342 

4-5 

**65. Newton’s laws: forces, 
reactions, velocity & acceleration 

5.42 

2.12 

5.5 

0.6686 

5-6 

**66. Summation of forces/force 
equilibrium 

5.00 

-1.52 

5 

0.6030 

5 

**67. Types of energy 

5.25 

0.37 

5 

0.6216 

5-6 

68. Circuit analysis and electrical 
power 

3.75 

-8.02 

4 

0.6216 

4 

69. Control theory 

2.83 

0.00 

3 

1.3371 

1.75-4 

70. Eluid fiow 

3.42 

-9.27 

4 

0.7930 

3-4 

71. Heat and mass balances 

3.58 

-8.75 

4 

0.7930 

3-4 

72. Heat transfer 

3.58 

-12.33 

4 

0.7930 

3-4 

**73. Statics 

4.50 

-10.00 

5 

1.000 

4-5 

**74. Strength of materials 

4.42 

3.85 

5 

1.0836 

4-5 

75. Thermodynamics 

3.33 

2.40 

3 

0.8876 

3-4 

76. Decision analysis 

3.67 

9.26 

3.5 

1.4355 

3-5 

77. Cognitive science 

3.08 

-14.81 

3 

1.1645 

2-4 

78. Teaming theories 

3.50 

-2.29 

3 

1.5076 

3-4.25 

79a. Project management 

4.25 

1.88 

4 

1.0553 

3.75-5 
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79b. Thermal expansion/contraction 

4.00 

2.00 

4 

0.8528 

3.75-4 

79c. Question asking -inquiry 

4.58 

N/A 

5 

0.7930 

4-5 

79d. Leadership principles 

3.58 

N/A 

4 

1.0836 

3.75-4 

79e. Principles related to 
environmental consciousness 

4.42 

N/A 

4.5 

0.6686 

4-5 

Research Question 4: What specific skills, techniques, and engineering tools related to 
engineering design should secondary students use to understand, manage, and solve 
technological problems? 

ITEM 

Mean 

A 

Mean 

(%) 

Median 

SD 

IQR 

80. Computer aided design 
software 

4.00 

-8.25 

4 

1.1282 

3.75-5 

**81. Computer searching 

4.67 

-5.35 

5 

0.8876 

4-5 

**82. E-mail 

5.18 

7.17 

5 

0.6030 

5-5.5 

83. Plotting Software 

4.50 

-2.50 

4.5 

0.7977 

4-5 

**84. Presentation software 

5.00 

3.17 

5 

0.7385 

4-5 

**85. Ability to abstract 

5.17 

-1.16 

5 

0.7177 

5-6 

**86. Ability to synthesize 

5.75 

1.01 

6 

0.4523 

5.75-6 

**87. Analogical reasoning 

5.17 

1.70 

5 

0.7177 

5-6 

88. Historical perspective 

4.42 

-0.93 

4.5 

1.1645 

4-5 

89. Analysis-based design 

4.17 

-1.92 

4 

1.2673 

4-5 

90. Basic mechanical mechanisms 

4.17 

-1.44 

4 

0.8348 

4-4.25 

91. Eailure mode and effects 
analysis 

3.08 

2.60 

3 

1.2401 

2.75-4 

**92. Product Dissection 

4.58 

-12.44 

5 

0.9962 

4-5 

93 a. Reverse engineering 

4.17 

0.00 

4 

1.1934 

4-5 

93b. Einishing job to the last 
detail 

3.67 

-6.81 

3.5 

1.4975 

3-4.25 

93 c. Recognizing team roles and 
personality types 

4.58 

0.07 

4.5 

0.6686 

4-5 

93 d. Engineering intuition 

3.45 

N/A 

4 

1.3685 

2.5-4.5 


Summary 

The Delphi technique was used for this research with the goal of adding to the growing 
body of literature on the subject of engineering design in secondary technology education 
courses. This study relied on four rounds to elicit the responses of persons considered to be 
experts in the field of engineering design. Participants accessed the survey instrument for each 


72 


round electronically via the Internet. A total of 88 unique items were identified during Round 1 
in answer to four open-ended research questions. In subsequent rounds participants suggested 
an additional 25 items for a total of 113 unique responses In Rounds 2 through 4 participants 
indicated their responses on six-point Likert scales. In Rounds 3 and 4, the statistical results from 
the previous round were reported to participants. 

The interquartile range and the inter-round mean score change were two major indices 
noted for each item in this study. The interquartile range indicated the degree of group consensus 
and the inter-round mean score change was an indication of item stability. After all four rounds 
were completed, eighty-four percent of the items (95/113) had achieved an IQR of <1. A total 
of 105 of the 113 items had measurable inter-round mean scores. The other eight were items 
suggested in Round 3 and were thus only included as survey items for one round, which was 
Round 4. Of thesel05 responses, 103 of them (98%) had a mean shift of <15 % at the end of all 
rounds. 

It was decided to identify items as very important for the purposes of this study only if 
they met 3 specific criteria: 

1 . An inter-round mean A of <1 5% (indicating stability) 

2. A median score of 5 or 6 (indicating a strong level of agreement among participants) 

3. An IQR range of < I (indicating consensus) 

Forty-eight items met these standards and are identified in Table 19 with double asterisk (**) 
symbols. These 48 items became the basis for the conclusions drawn from this Delphi process 
and the recommendations made for the field of technology education. Table 20 reports the 
ranking of each inidivual item by question. A “T” reported alongside the ranking for any item 
indicates that there was a tie between items identified for that question. Two items are listed 
as answers to two different research questions and should be understood in the context of the 
question they were intended to answer. The items are “Ability to handle open-ended problems” 
and “Multiple solutions to a single problem.” 
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Table 20 


Mean Score Ranking of Items Identified by Delphi Process 


Research Question 1 ; What aspects of the engineering design process best equip secondary 
students to understand, manage, and solve technological problems? 

ITEM 

Mean 

Rank (Ques.l) 

I . Understand problem identification/ formulation 
/development of requirements lists 

5.38 

5 

3. Understanding of customer needs 

5.00 

11 

5. Teamwork 

5.31 

6 

6. Decision making methodologies 

4.58 

17 

7. Written communication 

5.08 

4 

8. Oral communication 

5.54 

3 

9. Graphical/pictorial communication 

5.54 

15 

1 1 . Meeting skills 

4.62 

T16 

12. Personal ethics 

5.15 

T9 

14. Ability to break down complex problems 
in manageable pieces 

5.17 

8 

15. Ability to handle open-ended/ill defined problems 

5.77 

T1 

16. Ability to integrate multiple domains of knowledge 

5.08 

TIO 

17. Acceptance of multiple solutions to a single problem 

5.77 

T1 

18. Brainstorming and innovative concept generation 

5.15 

T9 

19. Conceptual design 

5.23 

TIO 

22. Experimental design, data 
collection, and interpretation of results 

4.54 

18 

23. Eunctional product modeling 

4.46 

19 

24. Human factors and safety in design 

4.62 

T16 

25. Identification of good/bad design 

4.62 

T16 

29. Prototyping/fabrication skills 

4.77 

T13 

3 1 . Research/library skills 

4.85 

T12 

32. Simplicity and clarity of use and function 

4.77 

T13 

33. Synthesis of simple parts into more complex system 

4.69 

14 

35. Critical thinking 

5.23 

7 

37. Logic and logical thinking 

4.85 

T12 

38. Systems thinking 

5.69 

2 

39d. Design principles to assist in generating innovative 
concepts 

4.67 

20 
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Research Question 2: What mathematics concepts related to engineering design should 

secondary students use to understand, manage, and solve technological problems? 

ITEM 

Mean 

Rank (Ques.2) 

40. Basic Algebra 

5.54 

T2 

4 1 . Advanced Algebra 

4.62 

T6 

43. Geometry 

5.46 

3 

44. Trigonometry 

5.00 

5 

45. Pre-Calculus 

4.62 

T6 

5 1 . Approximation 

4.54 

7 

52. Ability to handle open-ended/ill defined problems 

5.54 

T2 

53. Multiple solutions to a single problem 

5.69 

1 

56. Spreadsheets 

5.23 

4 

Research Question 3: What specific science principles related to engineering design should 

secondary students use to understand, manage, and solve technological problems? 

ITEM 

Mean 

Rank (Ques. 3) 

65. Newton’s laws: forces, reactions, velocity & acceleration 

5.42 

1 

66. Summation of forces/force equilibrium 

5.00 

3 

67. Types of energy 

5.25 

2 

73. Statics 

4.50 

4 

74. Strength of materials 

4.42 

5 

Research Question 4: What specific skills, techniques, and engineering tools related to 

engineering design should secondary students use to understand, manage, and solve 

technological problems? 



ITEM 

Mean 

Rank (Ques. 4) 

8 1 . Computer searching 

4.67 

5 

82. E-mail 

5.18 

2 

84. Presentation software 

5.00 

4 

85. Ability to abstract 

5.17 

T3 

86. Ability to synthesize 

5.75 

1 

87. Analogical reasoning 

5.17 

T3 

92. Product Dissection 

4.58 

6 
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CHAPTER 5 


SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS FOR FUTURE RESEARCH 

There has been a growing emphasis in the literature of technology education on the 
subject of engineering design (ITEA, 2000; ITEA, 2003; ITEA, 2005; Wicklein, 2006) . Some 
are currently looking to the field of engineering (and engineering design, specifically) as a 
focus for curriculum organization for the field of technology education (Lewis, 2004; Gorham, 
2002; Hailey, et al.2005). It is imperative that professionals in the field of technology education 
including teachers, administrators, and faculty at teacher preparation institutions have a firm 
grasp of the nature of engineering design and are properly equipped to merge engineering design 
content into the existing technology education curriculum. The literature of technology education 
has indicated a need for research in this area (Hailey, et ah, 2005; Pearson & Young; 2002 
Wicklein, 2006). 

The purpose of this study was to address some of those research needs for the field. 
Specifically, this study sought to use the Delphi process to elicit the responses of experts in the 
field of engineering design to four foundational research questions; 

1 . What aspects of the engineering design process best equip secondary students to 
understand, manage, and solve technological problems? 

2. What mathematics concepts related to engineering design should secondary students 
use to understand, manage, and solve technological problems? 

3. What specific science principles related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

4. What specific skills, techniques, and engineering tools related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 
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These research questions were designed to be open-ended and broad in order to help 
professionals in the field of technology education establish a framework for understanding 
engineering design in the context of secondary technology education classrooms. 

The Delphi process was considered to be an appropriate research methodology for the 
purposes of this study. The Delphi method is especially suited as a means of facilitating the 
interaction of a widely dispersed panel of participants. Participants in this study were identified 
by their peers through a logical, repeatable process. Many phases of this Delphi process 
including contacting participants and administering survey instruments were carried out via 
the Internet. A web-based research service was used to facilitate the process. Each survey was 
available online for 10 days, and participants received email reminders that instructed them to 
access the website in order to fill out the survey. 

This study utilized a four-round Delphi process in order to accomplish the goal of 
identifying items considered to be important answers to the four research questions. In Round I 
participants were asked to record 5-7 responses to each of the four research questions that were 
the focus of this study. A total of 234 responses were received. This data was reviewed by Dr. 
Paul Schrueders and Dr. Tim Taylor at Utah State University. Based on their review, a list of 88 
unique responses was established. These items became the basis for Round 2. 

The Round 2 survey instrument consisted of the 88 items identified in Round 1. For each 
individual item, participants were asked to indicate their level of agreement on a 6-point Likert 
scale. A higher score indicated a higher level of agreement that the item was an appropriate 
answer to the research question. In addition to indicating their level of agreement with each 
item, participants were free to add additional items in response to the research questions if they 
wished. 

The Round 3 survey instrument consisted of the original items identified in Round 1 plus 
15 items added in Round 2. Participants again accessed the survey electronically and indicated 
their level of agreement on a six-point Likert scale. In addition, the mean, standard deviation, 
maximum, minimum and interquartile range were reported alongside each item. Participants 
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were also emailed their Round 2 scores as a reminder. As in Round 2, participants were free to 
add any additional items they wished or to make comments. Specifically, participants were asked 
to provide an explanation if their score on any item was outside the IQR. 

As outlined in the Delphi literature, it was possible to drop some items after Round 3 and 
not include them in Round 4. Items that were included in Round 4 fell into one or more of the 
following categories: 

1 . Items that had a mean shift of >15% between Round 2 and Round 3 was considered to 
be unstable and were included in Round 4. 

2. Items with an Interquartile Range of >1 had not reached the level of consensus desired 
and was included in Round 4. 

3. Items which had been commented on during Round 3 were included in Round 4 along 
with the comments so that all participants could see their colleagues’ feedback. 

4. Items that were added in Round 3 were included in Round 4. 

Fifty items fell into one or more of these categories and were included in the Round 4 survey. 

The results of the Delphi process were analyzed and reported in Chapter 4 of this 
research. It was decided to identify items as very important only if they met high standards such 
as a median score of 5 or 6, IQR of < I and an inter-round A mean of < 15%. Forty-eight items 
met these standards and are identified in Table 19. 

General Summary 

As professionals in the field of technology education grapple with incorporating 
engineering design in secondary level classes, several conclusions can be drawn from this 
research. As the process of curriculum development moves forward, professionals in the field of 
technology education should make use of research-based content and instructional methodology 
in the creation of an overall curriculum framework for understanding and implementing 
engineering design. The development of a curriculum that emphasizes engineering design should 
be prefaced by the creation of a framework which provides insight from experts in the area of 
engineering design and extends the current Standards-based context of curriculum development. 
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Currently there is no overarehing framework for the understanding and implementation of 
engineering design content into secondary technology education classes. 

Therefore, the first conclusion to be drawn from this research is that the field is in need 
of a curriculum framework for the integration of engineering design in technology education 
classes. The creation and widespread acceptance of such a curriculum framework could help 
to bring a greater degree of solidarity to a fragmented assortment of approaches to the delivery 
of technology education courses currently practiced in high schools across the country. This 
overarching strategy of creating and implementing a solid engineering design focused curriculum 
framework is necessary to avoid a haphazard and disjointed experience for students and also for 
teachers attempting to use engineering design as a curriculum organizer. 

There are numerous approaches to the delivery of technology education content currently 
practiced in the United States, and this fragmented approach has led to confusion. It has also 
eroded the ability of the field to create a unified public image that would give technology 
education a greater degree of acceptance and infiuence among high school students, teachers, 
and parents. Technology teachers have indicated that they feel engineering design had a positive 
perception by the general public (Wicklein, 2004). Major stakeholders in the educational 
environment including administrators, teachers, parents, and students need to be able to clearly 
identify the goals and major activities associated with technology education. Incorporating 
engineering design into technology education and clearly articulating the learning outcomes, 
class activities, and related career opportunities could serve to improve the public perception 
of the field and thus alleviate many of the image problems that exist. 

Another conclusion to be drawn from this study is that integrating engineering design 
concepts into technology education classes could provide increased rigor as students apply 
academic skills and knowledge to technological problems. Career, technical, and agriculture 
education teachers are being encouraged to provide increased rigor in the curriculum and to 
emphasize the application of academic content where possible. Given this context, technology 
education would benefit greatly from the development of an engineering design focused 
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curriculum that features a logical progression in course content from elemental skills in 
introductory classes to advanced work involving the integration of concepts from mathematics 
and science in upper-level classes. 

Conclusions from Delphi Results 

A first and major conclusion to be made from the Delphi research process carried out as 
part of this research is that engineering design is a possible curriculum component for technology 
education courses that are specifically designed and reworked to incorporate this content area. 
Participants in this study were able to identify and indicate a high level of agreement with 48 
items that should be included in a technology education curriculum that emphasizes engineering 
design. This finding gives a strong indication that engineering design can in fact be considered 
as a potential contributor to the field of technology education. Professionals in the field of 
technology education should look seriously at the benefits of infusing the curriculum with 
content and methodology from the field of engineering design. It is therefore incumbent upon 
current technology teachers to seek out ways to educate themselves about engineering design and 
to seek out opportunities to learn more about an engineering design focused curriculum through 
professional development, additional coursework, etc. 

A second conclusion that can be made from the results of the Delphi study is that since 
survey items such as problem identification, solving open-ended problems, generating multiple 
solutions to a problem, the ability to synthesize, and systems thinking received the highest scores 
overall, an engineering design focused curriculum should emphasize these broad concepts. 

These findings have strong correlation to the Standards for Technological Literacy (ITEA, 2000) 
and other literature in the field that emphasizes problem solving and the ability to think broadly 
in the context of solving technological problems. This type of curriculum would be in direct 
contrast to a more structured pedagogy that demands wrote memory work without a great deal of 
higher-order thinking. 

An important consideration at this juncture is the current educational climate of 
accountability in which secondary technology education programs exist. Technology teachers 
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should clearly communicate the goals of their curriculum and the strategies employed so that 
parents, administrators, and counselors are aware of the traditionally academic content students 
make use of in technology education classrooms while solving technological problems. This can 
best be done through requiring students to carefully document and communicate their design 
process to others. This documentation work can be in the form of background research, written 
descriptions, hand sketches, computer-aided drawing (including 3D models), mathematical 
models, etc. Students should be required to develop potential solutions in the planning stages 
through careful calculation and modeling rather than through trial and error with actual 
components. Thus, teachers can display examples of student work so that stakeholders in the 
community can be made aware of the scope and nature of the technology education curriculum. 

A third conclusion is that various means of communication should also be emphasized 
since items that dealt with several different forms of communication also received high scores. 
Oral, written, and graphical communication all were emphasized by the participants and were 
deemed an extremely important component of engineering design. This finding again has 
correlation to literature in the field of technology education which specifically emphasizes 
the necessity of good communication in a variety of forms (ITEA, 2003). A project-oriented 
curriculum that emphasizes teamwork and communication would be best suited for teaching the 
engineering design process. 

A fourth conclusion from this study is that an engineering design-focused curriculum 
should emphasize teamwork and personal ethics. There was a high level of agreement that a 
secondary level technology education curriculum with an emphasis on engineering design should 
foster teamwork and interpersonal skills. It should also focus on the ethical responsibility of the 
designer to his or her fellow human beings. This finding somewhat contrasts with the typical 
instructional model that emphasizes the individual’s responsibility to perform independently 
on standardized tests. This approach is congruent with the literature in the field (ITEA, 2000; 
ITEA, 2003) which emphasize the importance of thinking broadly and looking for multiple 
points of view. 
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A fifth conclusion that can be drawn from this study is that the emphasis of a seeondary- 
level program should be on solving problems rather than on teaehing speeifie math/seience 
eoneepts to be applied at some time in the future. This eonelusion was evident beeause of 
some of the comments made by partieipants and also because of the very general nature of the 
responses to Questions Two and Three. At the outset of this study, it was thought that participants 
would identify many specific aspects of the various branches of mathematies and seience that 
are espeeially useful in design situations. However, participants focused on general topics such 
as algebra, geometry, ete. rather than providing detailed explanation of what specifically was 
most applicable from that area. The emphasis seemed to be on strueturing the currieulum so that 
students were required to make use of a wide range of knowledge in order to solve problems. 

This wide range of subjeet matter that may be eneountered in the eourse of solving 
teehnological problems is a very benefieial development because it naturally fosters inter- 
diseiplinary instruetion. Technology education teachers should seek out their colleagues in the 
diseiplines of mathematics and science in order to eollaborate on subject matter that might be 
unfamiliar. Collaboration with teachers from other disciplines can broaden the depth of the 
content for students, enrich the teaehers understanding of the related subjeet matter, and provide 
a more positive problem solving experience. 

A sixth conclusion from this study is that an engineering design-foeused eurriculum 
should inelude a hands-on eomponent because prototyping/fabrication skills reeeived high 
scores, as did product dissection. This finding fits well with typieal technology education 
practice. In a time when the hands-on component of the eurrieulum has been de-emphasized 
in some cireles, this result was strong evidenee that such training has an important place in 
the currieulum. Activities that emphasize modeling, fabrication, etc. tend to be higher interest 
for students and would help to create a contextual learning environment that would eneourage 
students to truly apply aeademie skills and knowledge in the proeess of creating solutions to 
teehnological problems. Carefully structured activities can be of high interest to students while 
requiring use of many of the mental processes (Halfin, 1973; Wicklein & Rojewski,1999), related 
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academic content, and concepts from engineering design. This type of learning environment 
would be greatly beneficial to students and would follow established contextual learning models 
(Parnell, 1995). 

Recommendations 

After completing this study and compiling the results, some recommendations can be 
made. These recommendations fall into three categories: future research, technology education 
instructional delivery methods, and teacher preparation. 

Recommendations for Future Research 

1 . Although this study sought to make a contribution to the development of a curriculum 

framework to be used in the delivery of course content in the area of technology education, 
further research is needed. There are specific aspects of using engineering design as a 
curriculum organizer that need to be addressed in order for the knowledge-base in this area 
to be complete enough to make this framework a reality. They are listed below: 

a. Academic content 

One of the issues to be addressed involves the general nature of the responses to the 
research questions in this study. Although the participants identified some general areas 
from mathematics and science that should be included in an engineering design-based 
curriculum, additional work needs to be done to determine what specific concepts from 
these areas are most applicable in the context of solving technological problems in 
technology education classes. For instance, algebra and geometry received high scores, 
but few specific concepts within these areas were identified. Further research should be 
done to determine what specific topics within these content areas are most applicable in 
the context of an engineering design-based curriculum. 

b. Pedagogy 

Another area needing future research involves the development of instructional methods 
designed to address the items identified as very important in the study. Since items 
such as solving open-ended problems, teamwork, and communication scored highly. 
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additional research is needed to determine how to best structure the curriculum in order 


to emphasize these skills. The literature has a good deal of information regarding many 
of these areas; however, more specific work should be done related to incorporate these 
specific concepts and skills from engineering design into the technology education 
curriculum. 

2. This study needs to be expanded by incorporating input from other engineering disciplines 
into the curriculum framework. Each participant in this research had a mechanical 
engineering background; additional studies might focus on other areas. Some engineering 
disciplines might be more suited to inclusion in this process than others are. Also, it would 
be beneficial to include more female and minority participants in future studies. 

Technology Education Instructional Delivery Methods 

3. The field should adopt a more unified approach centered on the topic of engineering design 
As stated earlier, there are several approaches to the delivery of curriculum content currently 
practiced in technology education classrooms across the country.. This does not mean that 
each class would be in lock-step sequence throughout the year, but rather that each program 
would use engineering design as its curriculum organizer. This one step would solve many 
of the issues facing the field and bring much needed direction to many who question the 
scope and purpose of technology education programs. 

4. The field of technology education should embrace content and concepts from engineering 
design. Classroom teachers should take the necessary steps to prepare themselves to 
integrate engineering design into their current curriculum. The results of the literature 
review completed for this study show that engineering design is similar in nature to the goals 
and intent of the Standards for Technological Literacy (ITEA, 2000). Many of the highest 
ranking items in this Delphi study are also important concepts in the Standards. Embracing 
concepts from engineering design and incorporating them into technology education courses 
could be a very positive development for the field. 
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Teacher Preparation 

This study has implications for teacher preparation institutions. Since many are currently 
looking to engineering design as a curriculum organizer and funding has been appropriated to 
form the NCETE, it would be beneficial for all programs that prepare technology education 
teachers to be aware of this development and take steps to prepare their students. This 
preparation involves at least three things: understanding of the engineering design process, 
developing the ability to facilitate classroom projects that enable students to engage in 
engineering design, and gaining the necessary academic skills to do so. 

5. The nature of engineering design could perhaps best be communicated to future teachers 
through requiring them to participate in classroom activities that emphasize engineering 
design. In other words, the projects would involve many of the items identified in this 
study. College students preparing to be technology teachers should be exposed to activities 
that involve open-ended problems, teamwork, and good communication using a variety of 
methods. Students should be required to use appropriate levels of mathematics and science 
in order to arrive at their solutions. In this way, future teachers of technology education will 
see the instructional method modeled before graduation and even be able to replicate some 
of the projects with their own students. 

6. Teacher preparation institutions should require their graduates to take appropriate 
mathematics and science courses. Some of these should come from the science and 
mathematics departments and others from technology education instructors. The latter 
should be focused on helping future technology teachers gain skills in applying mathematics 
and science in the context of various technological problem-solving activities. 

7. Teacher preparation institutions should make sure that faculty members are aware of the 
scope and nature of engineering design. This recommendation does not mean a total shift 
from the status quo. It does mean a greater awareness of engineering and, more specifically, 
a greater understanding of the engineering design process so that this aspect can be 
communicated to post-secondary students who are going into the classroom. 
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College of Education 

Workforce Education, Eeadership, and Social Eoundations 


Dr. Clive E. Dym 
Harvey Mudd College 
301 Platt Boulevard 
Claremont, C A 91711 

Dear Dr. Dym: 

My doctoral advisor. Dr. Robert Wicklein, has suggested that I contact you to request your help 
in establishing a pool of professors that may serve as experts in the field of engineering design 
for my doctoral research. If you agree to participate, you will be asked to identify the names 
and mailing addresses of 10 persons you feel are qualified as experts in the field of engineering 
design. These persons you identify will be asked to in-turn identify 10 individuals qualified to 
participate in the study. Erom this list, we will select approximately 25 participants for this study. 

My study is titled Engineering Design in Secondary Technology Education . The results 
of this study will help educators in the field of Technology Education develop a secondary 
curriculum that emphasizes the engineering design process. Many high schools across the 
country currently offer a program of study called Technology Education. Instruction typically 
centers on helping students achieve technological literacy through problem solving and design 
activities that utilize the technological problem solving method. Eeaders in the field have 
recognized that the engineering design process incorporates and concisely expresses many facets 
of what technology education seeks to equip students to do. However, there are many questions 
to be answered and this study seeks to contribute to the growing body of literature on the 
subject of how the engineering design process can be matriculated in the secondary technology 
education curriculum. 

You have been identified as my initial point of contact because of your extensive background and 
knowledge in engineering design and your highly respected international reputation. As you can 
see, the beginning of this entire study relies on your participation! I hope this does not assume 
too much, but we feel that your reputation and work with the Mudd Design Conference uniquely 
positions you to identify top experts in this field. 

This study will rely on the Delphi technique in order to elicit the responses of a geographically 
dispersed group in an anonymous and efficient manner. The study will be conducted in a series 
of four rounds - the online survey for each round should take less than 30 minutes to complete. 
Participants will access the study website and complete the survey one time for each successive 
round. 

Thank you for considering being involved in this study and we would ask that you please email 
the names and addresses of the 10 experts to me at the address below. If you are not able to 
do this at this time please advise me of this as well. I will be happy to answer any questions you 
might have- 1 hope to hear from you very soon! 
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Hello, as part of my doctoral study at the University of Georgia, I recently asked Dr. Clive 
Dym at Harvey Mudd College to name 12 persons he considered to be experts in the field of 
engineering design. You were one of those 12 - therefore I am contacting you at this time to ask 
you for the names of 12 persons that you feel are experts in the field of engineering design. My 
goal is to create a large pool of possible participants for my Delphi study. This study is related 
to developing a curriculum that emphasizes engineering design for students at the secondary 
level. If you will be so kind as to supply the name, school or company , and email address of 
12 individuals you feel would be very qualified to participate in this study it would be greatly 
appreciated and make progress possible. Your recommendation in no way obligates your 
colleagues! The study itself will be conducted via the Internet and will consist of 4 rounds of 
around 30 minutes each. 

Dr. Dym does not automatically assume you will have time to supply the list of names, but he 
does feel that you are qualified. This is a crucial step in my study and I can not go forward until I 
gain this list. I can supply more details about the logistics of the study on request and feel free to 
contact me if you have any questions. Thank you for your consideration! 

Cameron Smith 
Doctoral Candidate 
University of Georgia 
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Round 1 Survey 


1 . What aspects of the engineering design process best equip secondary students to 
understand, manage, and solve technological problems? 

2. What mathematics concepts related to engineering design should secondary students 
use to understand, manage, and solve technological problems? 

3. What specific science principles related to engineering design should secondary 
students use to understand, manage, and solve technological problems? 

4. What specific skills, techniques, and engineering tools related to engineering design 
should secondary students use to understand, manage, and solve technological 
problems? 
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Participant ‘A’ 

Q 1 .Understanding the problem, understanding customer needs, brainstorming solutions You ean 
get pretty far with algebra for a number of problems (statics, some fluids, ete. 

Q2. Should understand eonservation of energy and momentum, conversion of energy to different 
types (potential, kinetie, etc.) Force equilibrium for statie problems, F = ma 

Q3. Identifying the underlying scientific principles governing a problem, Basie physics 

Q4. Learning how to And speeialized knowledge. Understanding how to set up a program 
of experiments and evaluate results Brainstorming skills to determine lots of coneepts 
Communieation skills Shop fabrication (or access to such skills) drawing skills (possibly 
CAD skills) 


Participant ‘B’ 

Q 1 . 1 . . Ability to thi nk about open ended, ill-defined problems. Mueh of seeondary sehool 
math/scienee education focuses on solving problems to get a single right answer, and engineering 
design process helps them think more broadly. 2. Need to synthesize different types of 
knowledge. Engineering design process eneourages individuals to draw upon different types of 
knowledge all at once, rather than one discipline. 

Q2. Seeondary students need to know be familiar with basie engineering math, whieh means 
algebra and geometry, and if possible, calculus. 

Q3. Basic physics is important, in particular introductory mechanics. 

Q4. The proeess of generating ideas, prototyping them, and testing them is key in doing 
engineering design at any level. For seeondary school students, this might require teaehing 
students some prototyping skills (freehand drawing, building ideas in soft materials, or working 
with simple hand- or even maehine -tools). 

Partieipant ‘C’ 

Ql. Identifying a need, requirements list, eoneeptual design, computer-aided modeling 
Q2. ealeulus, algebra, linear algebra, statisties 
Q3. physies, chemistry, biology 

Q4. problem solving, idea generation, CAD, prototyping, eomputer programming 
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Participant ‘D’ 

Q 1 . Physics based approaches based on conservation laws 

Q2. Optimization - particularly evolutionary optimization In-Out = change on a integral basis 
Q3. summation of forces = mass x acceleration 

Q4. A strong command of algebra, and an ability to use common engineering software tools. 


Participant’E’ 

Q1 . recognition and formulation of design problems, generate potential design alternative 
evaluation of alternatives data collection and interpretation decision making 

Q2. measurement theory optimization statistics probability 

Q3. science principles rooted in the above mathematics concepts Understand the problem 
formulate a problem generate design concepts provide mathematical descriptions of design 
solutions 

Q4. Search design solution using efficient computer algorithms design experiments and analyze 
data apply decision making principles communication team work synthesis dealing with open- 
ended problems 

Participant ‘F’ 

Ql. open-ended problem solving, working prototype construction, use of basic physics, 
chemistry, math applied to analysis of a design situation 

Q2. algebra, statistics, simple programming 

Q3. physics: particularly mechanics, simple circuit analysis chemistry: as it relates to material 
properties and manufacturing possibilities 

Q4. The key idea of the engineering method that you must solve a problem in the face of 
incomplete information; be able to abstract the real problem to an appropriate level of abstraction 
so you can apply useful analysis from physics etc. This issue is the biggest problem students 
face all the way to finishing their Ph.D.s. 

Participant ‘G’ 

Ql. The simple answer is all of them (this is too open a question). Some specifics include: 
Problem identification. Understanding the customer wants. 

Q2. Writing a quantitative specification. Too broad a question. Algebra, trigonometry, pre- 
calculus 
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Q3. Too broad a question. It depends on the problem being solved: physies, chemistry, biology, 
mathematics. 

Q4. Too broad a question. Drafting, machining, wood shop. 

Participant ‘H’ 

Ql. Brainstorming; intuition; hands-on experience dissecting products, etc.; critical thinking 
skills; project planning skills; good oral and written communication skills; artistic talent/ 
sketching ability; ability to reason and debate good and bad merits of a design concept. 

Q2. This is somewhat problem-dependent: some problems require little to no math while others 
require complex analysis using matrices, differential equations, etc. I think what is more critical 
is knowing how to set up, mathematically model, and formulate the problem at hand and then 
knowing what math skills are needed - and where to find, say, relevant equations - to solve the 
problem. 

Q3. Same thing as with math skills, i.e., very problem-dependent. Some problems require simple 
laws like F=ma while others are much more complex (e.g., laws of fiuid fiow, thermodynamics, 
etc.). Again, what is important is being able to reason critically about the problem to identify 
what types of principles are needed and then knowing where to find them. 

Q4. Same answer as Ql. 

Participant T’ 

Ql. to know there is a process and understand the major steps and tools available nd used in 
industry. 

Q2. To experience them doing a project - not garage engineering. 

Q3. There is virtually no math needed. If anything a good basis of statistics as all design is 
uncertain 

Q4. Design process is not science. Engineering analysis is based on science and is a necessary 
part of the design process. QFD, Pugh’s method, FMEA, and all the rest I cover in “The 
Mechanical Design Process” 

Participant ‘J’ 

Ql. Logical thinking. 

Q2. Logical thinking. 

Q3. Mathematics (just fundamental algebra and geometry), 

Q4. physics (I guess I am biased toward machine design). 

Logical thinking, a bit of commonsense. 
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Participant ‘K’ 

Ql. Within the scope of secondary school education; * Understanding requirements of problem 
in context * Working as a goal-oriented team, making best use of resources/talent * Developing 
multiple concepts for meeting specified requirements * Selection of most appropriate concept 
within overall context * Developing chosen concept into a practicable solution * Presenting the 
solution in a convincing manner 

Q2. Some mathematical concepts useful for underpinning student interaction with technological 
problems: * The concept of dealing with approximations rather than exact numbers * The 
concept of finding solutions without all the necessary information * The concept of mapping 
alternatives in different ways * The concept of statistical exploration of life-cycle issues 

Q3. Some scientific “principles” important to understand as an underpinning to the solution 
of technological problems: * Force and deflection - visualization of force transmission paths 
* Stress and distortion - visualization of material capabilities * Expansion and contraction - 
thermal effects on materials * Heat transfer - conduction, convection and radiation * Energy 
conversion - mass balance * Static pressure and fluid flow - visualization and general concepts * 
Basics of electrical power theory and electronic controls 

Q4. Some skills, techniques and engineering tools that could be useful in the context of 
encouraging students to increase their technology involvement: * Genuine understanding of 
personal capabilities and limitations * Enthusiasm and respect for historical/prior technological 
developments * Excellence in negotiation and communication - both oral and written * 

High degree of cross-cultural understanding and multiple language skills * Understanding of 
ethical issues and effects of external influences * Ability to visualize and describe problems in 
diagrammatic form * Effective use of knowledge and resources to finish a task completely 


Participant ‘E’ 

Ql. Understand; Identification for problem via function structures. Manage: Creation of a work 
schedule Solve: Recognition that the solution method depends on the type of problem at hand. 

Q2. Physics 

Q3. This is too broad a question to elicit a meaningful response. In general principles of 
conservation of mass and energy; Newton’s laws 

Q4. Ability to abstract. Ability to analyze. Ability to synthesize. Ability to evaluate design so 
that it can be improved. 


Participant ‘M’ 

Ql. Ability to break down complex problems in manageable pieces. Synthesis of simple parts 
into more complex system. Eirst hand learning and experience of how products, parts, and 
engineered systems in general work (and fail). Teamwork. 
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Q2. Integration, differentiation, algebra (solving systems of equations), trigonometry. Numerieal 
problem solving using computers and spreadsheets. 

Q3. Energy principles (potential, kinetic, heat, etc). Newton’s Laws: Forces, reactions, velocity 
& acceleration Some basic strength, strain & stress principles (tension, compression, shear, 
torsion - what is it & what can it do) 

Q4. Some basic material selection principles (steel vs iron vs aluminum vs plastic, etc). Some 
basic understanding of standard components (what is a roller bearing, gear, shaft, electromotor, 
etc) 

Good general computer skills (word processing, presentation, e-mail). Especially well versed 
in using spreadsheets, including graphing. Good research skills (how to find info on something 
you don’t know much about and document your findings), using Internet/web, library, personal 
contacts/sources. Teamwork, planning, and meeting skills. Some solid modeling CAD skills to 
show the final product, e.g., using SolidEdge (do NOT use AutoCAD - that is outdated for most 
engineers, except civil engineers). 

Participant ‘N’ 

Ql. Aspects include innovative concept generation, functional modeling, customer needs 
gathering and problem clarification, and product modeling. 

Q2. Concepts include analytical geometry, trigonometry, advanced algebra, and basic calculus 
and differential equations. 

Q3. General principles include robustness, clarity (functional independence), simplicity (minimal 
information content), and safety (human factors). More specific principles are the Theory of 
Inventive Problem Solving Laws of Evolution and Design Principles. 

Q4. Skills: Problem Solving, problem reformulation, fabrication in various materials, modeling 
of physical systems; Techniques: Analogical Reasoning, functional modeling, quality function 
deployment, team-based synthesis (such as 6-3-5). 

Participant ‘O’ 

Q 1 . Understanding context, managing ambiguity, identifying needs, constructing requirements, 
scientific experimentation and prototyping, defining and refining intent, systems thinking. 

Q2. Probability, statistics, numerical simulation, and the obvious basic engineering math such as 
calculus and linear algebra. 

Q3. Basic understanding of dynamic systems, circuits, strength of materials, thermodynamics, 
decision analysis, cognitive science, learning theories. 

Q4. CAD tools (visual and numerical), sketching, prototyping and manufacturing methods, 
ethnography (qualitative observation of people/users), communication (as in interacting with 
users/team members and presenting/documenting work), negotiation. 
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Round Two Survey 

What is your Respondent ID number? 

The following 38 items are responses to Question 1: What aspects of the engineering design 
process best equip secondary students to understand, manage, and solve technological 
problems? For each item, please indicate your level of agreement with the following statement; 
This item is a critical and necessary component of an engineering design focused curriculum 
which is designed to equip secondary (high school) students to understand, manage, and solve 
technological problems. 

1 . Understand problem identification / formulation / development of requirements lists 

Strongly Disagree 

P 

Disagree 

P 

Somewhat Disagree 

P 

Somewhat Agree 

P 

Agree 

P 

Strongly Agree 


2. Understand functional structures 

P 

Strongly Disagree 

P 

Disagree 

P 

Somewhat Disagree 
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Note - the remaining items in this survey instrument have the Likert scale removed to make this 
document more readable. 

3. Understanding of customer needs 

4. Project planning and scheduling 

5. Teamwork 

6. Decision making methodologies 

7. Written communication 

8. Oral communication 

9. Graphical/pictorial communication 

10. Negotiation 

1 1 . Meeting skills 

12. Personal ethics 

13. Multicultural/diversity awareness 

14. Ability to break down complex problems in manageable pieces 

15. Ability to handle open-ended/ill-defined problems 

16. Ability to integrate multiple domains of knowledge 

17. Acceptance of multiple solutions to a single problem 

18. Brainstorming and innovative concept generation 

19. Conceptual design 

20. Design for robustness/failure mode analysis 

21. Engineering heuristics for/analysis-based design 

22. Experimental design, data collection, and interpretation of results 

23. Eunctional product modeling 

24. Human factors and safety in design 

25. Identification of good/bad design 

26. Identification of underlying scientific principles 

27. Product optimization 

28. Product testing/functional analysis 

29. Prototyping/fabrication skills 

30. Recognition that the solution method depends on the type of problem at hand 

3 1 . Research/library skills 

32. Simplicity and clarity of use and function 

33. Synthesis of simple parts into more complex system. 

34. Understanding product life cycles/life cycle analysis 

35. Critical thinking 

36. Experience 

37. Logic and logical thinking 

38. Systems thinking 

39. Can you list other aspects of the engineering design process that would best equip 
secondary students to understand, manage, and solve technological problems? 
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The next 18 items are responses to Question 2: What mathematics concepts related to 
engineering design should secondary students use to understand, manage and solve technological 
problems? For each item below, please indicate your level of agreement with the following 
statement; This item is a critical and necessary component of an engineering design focused 
curriculum designed to require secondary (high school) students to use various mathematical 
concepts to understand, manage, and solve technological problems. 

40. Basic Algebra 

41. Advanced Algebra 

42. Linear Algebra 

43. Geometry 

44. Trigonometry 

45. Pre-Calculus 

46. Statistics 

47. Calculus- Integration 

48. Calculus- Differentiation 

49. Calculus- Differential Equations 

50. Measurement theory 

51. Approximation 

52. Ability to handle open-ended/ill-defined problems 

53. Multiple solutions to a single problem 

54. Optimization 

55. Computer Programming 

56. Spreadsheets 

57. Modeling/simulation/numerical analysis software 

58. Can you provide specific formulas or specific mathematics concepts commonly 
used in engineering design that would be useful for secondary (high school) students 
involved in understanding, managing, and solving technological problems? 

The next 20 items are responses to Question 3: What specific science principles related to 
engineering design should secondary students use to understand, manage and solve technological 
problems? For each item below, please indicate your level of agreement with the following 
statement; This item is a critical and necessary component of an engineering design focused 
curriculum designed to require secondary (high school) students to use various scientific 
principles to understand, manage, and solve technological problems. 

59. Chemical properties of materials 

60. Effects of chemical formulation on manufacturing 

6 1 . Biological Evolution 

62. Conservation of mass, energy, and momentum 

63. Dynamic systems 

64. Introductory mechanics 

65. Newton’s laws; forces, reactions, velocity & acceleration 

66. Summation of forces/force equilibrium 

67. Types of energy 
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68. Circuit analysis and electrical power 

69. Control theory 

70. Fluid flow 

7 1 . Heat and mass balanees 

72. Heat transfer 

73. Statics 

74. Strength of materials 

75. Thermodynamies 

76. Deeision analysis 

77. Cognitive scienee 

78. Learning theories 

79. Can you provide other seientiflc prineiples that should be emphasized in an 
engineering design focused currieulum that involves seeondary (high school) students in 
understanding, managing, and solving technologieal problems? 

The next 13 items are responses to Question 4: What speeifle skills, teehniques, and engineering 
tools related to engineering design should seeondary students use to understand, manage and 
solve teehnological problems? For each item below, please indicate your level of agreement with 
the following statement: This item is a eritieal and necessary eomponent of an engineering design 
focused eurrieulum designed to require secondary (high school) students to use various skills, 
teehniques, and engineering tools to understand, manage, and solve technologieal problems. 

80. Computer aided design software 

8 1 . Computer searching 

82. E-mail 

83. Plotting Software 

84. Presentation software 

85. Ability to abstract 

86. Ability to synthesize 

87. Analogical reasoning 

88. Historieal perspeetive 

89. Analysis-based design 

90. Basie meehanical meehanisms 

91 . Failure mode and effects analysis 

92. Product Dissection 

93. Can you list additional skills, techniques, or engineering tools related to engineering 

design that seeondary (high school) students should use to understand, manage, and solve 
teehnological problems? 
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Thank you for your participation thus far in the Engineering Design in Technology Education 
study! As you know, Rounds One and Two are complete and I have been compiling all the 
responses and preparing Round Three. This has taken quite a bit of time and was more of a 
challenge than I antieipated! In preparation for beginning Round Three, I am sending out this 
instructional email - I will send another email later today that contains the link to the Round 
Three survey. Please note the following three items: 

1 . This study is designed to help in the ereation of a framework for a eurriculum which 
emphasizes engineering design for high school students. Please keep in mind the level 
of math/scienee courses in whieh high sehool students are normally enrolled (a small 
minority of students may take Caleulus during the 12* grade; a larger number may have 
Physics during the 11* or 12 grade). 

2. There are eomment blanks at the end of each section. Please take a moment to eonsider 
any speeific answers that eome to mind. These would be more helpful to actually creating 
a eurrieulum framework than more general topics such as algebra, etc. 

3. This study is utilizing the Delphi method. This method dictates that in Round Three 
participants be given the Round Two survey items again - along with statistical data 
which indicates the group response to each item. I acknowledge that this seems 
redundant. However, if you will bear with the proeess, the goal is to give you an 
opportunity to eonsider your choiees again in light of the statistieal data. This may lead 
to a change in your answers from Round Two or it may not. To assist you in this proeess 
I have attached an Exeel file to this email with your responses to the Round Two items . 
Please let me know if you have trouble opening the file. 

Again, tha nk you very mueh for making this study possible! 

Cameron Smith 
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What is your Respondent ID number? 


The following section consists of responses to Question 1 : What aspects of the engineering 
design process best equip secondary students to understand, manage, and solve 
technological problems? For each item , please indicate your level of agreement with the 
following statement: This item is a critical and necessary component of an engineering 
design focused curriculum which is designed to equip secondary (high school) students to 
understand, manage, and solve technological problems. *The Round Two data is included 
with each question. Please make note of whether your answer falls outside the Interquartile 
Range (IQR) or middle 50%. If it does, please feel free to comment as to why you made this 
choice in the comments box at the end of this survey. 


1 . Understand problem identification / formulation / development of requirements lists 
(Round Two Data: Mean= 5.85, Max=6, Min=5, St. Dev.= 0.3755, IQR= 6) 

P 

Strongly Disagree (1) 

^ Disagree (2) 

P 

Somewhat Disagree (3) 

P 

Somewhat Agree (4) 

^ Agree (5) 

P 

Strongly Agree (6) 

2. Understand functional structures (Round Two Data: Mean= 4.77, Max=6, Min=2, St. 
Dev.= 1.3634, IQR= 4-6) 

P 

Strongly Disagree (1) 

^ Disagree (2) 

P 

Somewhat Disagree (3) 

P 

Somewhat Agree (4) 

^ Agree (5) 

P 

Strongly Agree (6) 
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Note - the additional items from the Round Three survey instrument are displayed here without 
the associated Likert scale to make this document more readable. 


3. Understanding of customer needs (Round Two Data: Mean= 5.31, Max=6, Min=3, St. 
Dev.= 0.8549, IQR= 5-6) 

4. Project planning and scheduling (Round Two Data: Mean= 4.92, Max=6, Min=4, St. 
Dev.= 0.7596, IQR= 4-5) 

5. Teamwork (Round Two Data: Mean= 5.23, Max=6, Min=4, St. Dev.= 0.5991, IQR= 5-6) 

6. Decision making methodologies (Round Two Data: Mean= 4.62, Max=6, Min=3, St. 
Dev.= 0.1.0439, IQR= 4-5) 

7. Written communication (Round Two Data: Mean= 5.23, Max=6, Min=4, St. Dev.= 
0.8321, IQR= 5-6) 

8. Oral communication (Round Two Data: Mean= 5.54, Max=6, Min=5, St. Dev.= 0.5189, 
IQR= 5-6) 

9. Graphical/pictorial communication (Round Two Data: Mean= 5.23, Max=6, Min=4, St. 
Dev.= 0.7250, IQR= 5-6) 

10. Negotiation (Round Two Data: Mean= 4.15, Max=6, Min=3, St. Dev.= 0.8006, IQR= 4) 

11. Meeting skills (Round Two Data: Mean= 4.54, Max=6, Min=4, St. Dev.= 0.6602, IQR= 
4-5) 

12. Personal ethics (Round Two Data: Mean= 5.00, Max=6, Min=4, St. Dev.= 0.7071, IQR= 

5) 

13. Multicultural/diversity awareness (Round Two Data: Mean= 4.38, Max=6, Min=2, St. 
Dev.= 1.2609, IQR= 4-5) 

14. Ability to break down complex problems in manageable pieces (Round Two Data: Mean= 
5.00, Max=6, Min=4, St. Dev.= 0.8165, IQR= 4-6) 

15. Ability to handle open-ended/ill defined problems (Round Two Data: Mean= 5.46, 
Max=6, Min=4, St. Dev.= 0.6602, IQR= 5-6) 

16. Ability to integrate multiple domains of knowledge (Round Two Data: Mean= 5.31, 
Max=6, Min=3, St. Dev.= 0.9473, IQR= 5-6) 

17. Acceptance of multiple solutions to a single problem (Round Two Data: Mean= 5.62, 
Max=6, Min=5, St. Dev.= 0.5064, IQR= 5-6) 

18. Brainstorming and innovative concept generation (Round Two Data: Mean= 5.00, 

Max=6, Min=4, St. Dev.= 0.5774, IQR= 5) 

19. Conceptual design (Round Two Data: Mean= 5.42, Max=6, Min=5, St. Dev.= 0.5149, 
IQR= 5-6) 

20. Design for robustness/failure mode analysis (Round Two Data: Mean= 3.85, Max=5, 
Min=l, St. Dev.= 1.0682, IQR= 4) 

21. Engineering heuristics for/analysis-based design (Round Two Data: Mean= 3.92, Max=6, 
Min=5, St. Dev.= 1.3790, IQR= 3.75-5) 

22. Experimental design, data collection, and interpretation of results (Round Two Data: 
Mean= 4.46, Max=6, Min=3, St. Dev.= 0.7763, IQR= 4-5) 

23. Eunctional product modeling (Round Two Data: Mean= 4.31, Max=6, Min=3, St. Dev.= 
0.8549, IQR= 4-5) 
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24. Human factors and safety in design (Round Two Data: Mean= 4.54, Max=5, Min=4, St. 
Dev.= 0.5189, IQR= 4-5) 

25. Identification of good/bad design (Round Two Data: Mean= 4.77, Max=6, Min=4, St. 
Dev.= 0.7250, IQR= 4-5) 

26. Identification of underlying scientific principles (Round Two Data: Mean= 4.77, Max=6, 
Min=3, St. Dev.= 0.8321, IQR= 4-5) 

27. Product optimization (Round Two Data: Mean= 3.54, Max=5, Min=2, St. Dev.= 0.8771, 
IQR= 3-4) 

28. Product testing/functional analysis (Round Two Data: Mean=4.46 , Max=6, Min=3, St. 
Dev.=0.8771 , IQR=4-5) 

29. Prototyping/fabrication skills (Round Two Data: Mean=4.85 , Max=6, Min=4, St. 
Dev.=0.8006 , IQR=4-5) 

30. Recognition that the solution method depends on the type of problem at hand (Round 
Two Data: Mean=4.31 , Max=6, Min=3, St. Dev. =0.8549 , IQR=4-5) 

31. Research/library skills (Round Two Data: Mean=4.62 , Max=6, Min=2, St. Dev.=1.0439 , 
IQR=4-5) 

32. Simplicity and clarity of use and function (Round Two Data: Mean=4.54 , Max=6, 

Min=4, St. Dev.=0.6602 , IQR=4-5) 

33. Synthesis of simple parts into more complex system. (Round Two Data: Mean=4.54 , 
Max=6, Min=3, St. Dev.=0.7763 , IQR=4-5) 

34. Understanding product life cycles/life cycle analysis (Round Two Data: Mean=4.38 , 
Max=6, Min=3, St. Dev.=0.7679 , IQR=4-5) 

35. Critical thinking (Round Two Data: Mean=5.23 , Max=6, Min=4, St. Dev.=.08321 , 
IQR=5-6) 

36. Experience (Round Two Data: Mean=4.00 , Max=6, Min=2, St. Dev.=1.2910 , IQR=3-5) 

37. Logic and logical thinking (Round Two Data: Mean=4.92 , Max=6, Min=4, St. 
Dev.=0.6405 , IQR=5) 

38. Systems thinking (Round Two Data: Mean=5.31 , Max=6, Min=4, St. Dev. =0.8549 , 
IQR=5-6) 

39. Can you list other aspects of the engineering design process that would best equip 
secondary students to understand, manage, and solve technological problems? 

39a. Costing, profit, and basic economic analysis (new item, no Round Two data) 

39b. Understanding the context of the technological problem and possible external influences 
(new item, no Round Two data) 

39c. Product architecture and modularity/interfaces (new item, no Round Two data) 

39d. Design principles to assist in generating innovative concepts (new item, no Round Two 
data) 

39e. Design by analogy (new item, no Round Two data) 

The next section consists of responses to Question 2: What mathematics concepts related to 
engineering design should secondary students use to understand, manage and solve technological 
problems? For each item below, please indicate your level of agreement with the following 
statement: This item is a critical and necessary component of an engineering design focused 
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curriculum designed to require secondary (high school) students to use various mathematical 
concepts to understand, manage, and solve technological problems. *The Round Two data 
is included with each question. Please make note of whether your answer falls outside the 
Interquartile Range (IQR) or middle 50%. If it does, please feel free to comment as to why you 
made this choice in the comments box at the end of this survey. 

40. Basic Algebra (Round Two Data: Mean=5.38 , Max=6, Min=4, St. Dev. =0.7679 , 
IQR=5-6) 

41. Advanced Algebra (Round Two Data: Mean=4.85 , Max=6, Min=2, St. Dev.=1.1435 , 
IQR=4-6) 

42. Linear Algebra (Round Two Data: Mean=4.00 , Max=5, Min=2, St. Dev.=1.0000 , 
IQR=3-5) 

43. Geometry (Round Two Data: Mean=5.15 , Max=6, Min=4, St. Dev. =0.6887 , IQR=5-6) 

44. Trigonometry (Round Two Data: Mean=5.17 , Max=6, Min=4, St. Dev. =0.7 177 , 
IQR=5-6) 

45. Pre-Calculus (Round Two Data: Mean=4.75 , Max=6, Min=4, St. Dev. =0.6216 , 
IQR=4-5) 

46. Statistics (Round Two Data: Mean=4.85 , Max=6, Min=4, St. Dev. =0.8987 , IQR=4-6) 

47. Calculus- Integration (Round Two Data: Mean=4.62 , Max=6, Min=3, St. Dev.=.9608 , 
IQR=4-5) 

48. Calculus- Differentiation (Round Two Data: Mean=4.62 , Max=6, Min=3, St. Dev.=.9608 
, IQR=4-5) 

49. Calculus- Differential Equations (Round Two Data: Mean=3.92 , Max=5, Min=2, St. 
Dev.=0.7596 , IQR=4) 

50. Measurement theory (Round Two Data: Mean=4.23 , Max=6, Min=3, St. Dev. =0.7250 , 
IQR=4) 

51. Approximation (Round Two Data: Mean=5.08 , Max=6, Min=3, St. Dev.=1.0377 , 
IQR=4-6) 

52. Ability to handle open-ended/ill defined problems (Round Two Data: Mean=5.62 , 
Max=6, Min=4, St. Dev.=0.6504 , IQR=5-6) 

53. Multiple solutions to a single problem (Round Two Data: Mean=5.46 , Max=6, Min=4, 

St. Dev.=.6602 , IQR=5-6) 

54. Optimization (Round Two Data: Mean=3.31 , Max=5, Min=l, St. Dev.=1.1094 , 
IQR=3-4) 

55. Computer Programming (Round Two Data: Mean=4.23 , Max=6, Min=2, St. 
Dev.=1.0919,IQR=4-5) 

56. Spreadsheets (Round Two Data: Mean=5.15 , Max=6, Min=4, St. Dev. =0.8006 , 
IQR=5-6) 

57. Modeling/simulation/numerical analysis software (Round Two Data: Mean=4.46 , 
Max=6, Min=3, St. Dev.=0.9674 , IQR=4-5) 

58. Can you provide specific formulas or specific mathematics concepts commonly used in 
engineering design that would be useful for secondary (high school) students involved in 
understanding, managing, and solving technological problems? 
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58 a. Algebraic equations for determining gear ratios (new item, no Round Two data) 

58b. Conservation of momentum (new item, no Round Two data) 

58c. Projectile motion (new item, no Round Two data) 

58d. Structural equilibrium (new item, no Round Two data) 

58c. Basic stresses (new item, no Round Two data) 

The next section consists of responses to Question 3: What specific science principles 
related to engineering design should secondary students use to understand, manage 
and solve technological problems? For each item below, please indicate your level of 
agreement with the following statement: This item is a critical and necessary component 
of an engineering design focused curriculum designed to require secondary (high 
school) students to use various scientific principles to understand, manage, and solve 
technological problems. *The Round Two data is included with each question. Please 
make note of whether your answer falls outside the Interquartile Range (IQR) or middle 
50%. If it does, please feel free to comment as to why you made this choice in the 
comments box at the end of this survey. 

59. Chemical properties of materials (Round Two Data: Mean=4.17 , Max=5, Min=3, St. 
Dev.=0.5774 , IQR=4-4.25) 

60. Effects of chemical formulation on manufacturing (Round Two Data: Mean=3.38 , 
Max=5, Min=2, St. Dev.=1.0439 , IQR=3-4) 

61. Biological Evolution (Round Two Data: Mean=3.31 , Max=6, Min=l, St. Dev.=1.3775 , 
IQR=2-4) 

62. Conservation of mass, energy, and momentum (no Round Two data available) 

63. Dynamic systems (Round Two Data: Mean=4.23 , Max=5, Min=3, St. Dev. =0.5991 , 
IQR=4-5) 

64. Introductory mechanics (Round Two Data: Mean=4.77 , Max=6, Min=4, St. 

Dev.=0.7250, IQR=4-5) 

65. Newton’s laws: forces, reactions, velocity & acceleration (Round Two Data: Mean=5.31 , 
Max=6, Min=4, St. Dev.=0.6304 , IQR=5-6) 

66. Summation of forces/force equilibrium (Round Two Data: Mean= 5.08 , Max=6, Min=4, 
St. Dev.=0.6405 , IQR=5) 

67. Types of energy (Round Two Data: Mean=5.23 , Max=6, Min=4, St. Dev. =0.5991 , 
IQR=5-6) 

68. Circuit analysis and electrical power (Round Two Data: Mean=4.08 , Max=5, Min=3, St. 
Dev.=0.4935 , IQR=4) 

69. Control theory (Round Two Data: Mean=3.00 , Max=5, Min=2, St. Dev.=1.0000 , 
IQR=2-4) 

70. Eluid flow (Round Two Data: Mean=3.77 , Max=5, Min=3, St. Dev. =0.5991 , IQR=3-4) 

71. Heat and mass balances (Round Two Data: Mean=3.92 , Max=5, Min=3, St. 
Dev.=0.7596, IQR=3-4) 

72. Heat transfer (Round Two Data: Mean=4.08 , Max=6, Min=3, St. Dev. =0.7930 , IQR=4) 

73. Statics (Round Two Data: Mean=5.00 , Max=6, Min=4, St. Dev. =0.7385 , 
IQR=4.75-5.25) 
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74. Strength of materials (Round Two Data: Mean=4.69 , Max=6, Min=3, St. Dev. =0.9473 , 
IQR=4-5) 

75. Thermodynamics (Round Two Data: Mean=3.85 , Max=6, Min=2, St. Dev.=1.0682, 
IQR=3-4) 

76. Decision analysis (Round Two Data: Mean=3.77 , Max=6, Min=2, St. Dev.=1.4233 , 
IQR=3-4) 

77. Cognitive science (Round Two Data: Mean=3.62 , Max=6, Min=2, St. Dev.=1.3868 , 
IQR=2-5) 

78. Learning theories (Round Two Data: Mean=3.62 , Max=6, Min=2, St. Dev.=1.5021 , 
IQR=3-4) 

79. Can you provide other scientific principles that should be emphasized in an engineering 
design focused curriculum that involves secondary (high school) students in 
understanding, managing, and solving technological problems? 

79a. Project management (new item, no Round Two data) 

79b. Thermal expansion/contraction (new item, no Round Two data) 

The last section consists of responses to Question 4: What specific skills, techniques, and 
engineering tools related to engineering design should secondary students use to understand, 
manage and solve technological problems? For each item below, please indicate your level of 
agreement with the following statement: This item is a critical and necessary component of an 
engineering design focused curriculum designed to require secondary (high school) students 
to use various skills, techniques, and engineering tools to understand, manage, and solve 
technological problems. *The Round Two data is included with each question. Please make note 
of whether your answer falls outside the Interquartile Range (IQR) or middle 50%. If it does, 
please feel free to comment as to why you made this choice in the comments box at the end of 
this survey. 

80. Computer aided design software (Round Two Data: Mean=4.46 , Max=6, Min=3, St. 
Dev.=0.8771 , IQR=4-5) 

81. Computer searching (Round Two Data: Mean=4.38 , Max=6, Min=2, St. Dev.=1.3253 , 
IQR=4-5) 

82. E-mail (Round Two Data: Mean=4.83 , Max=6, Min=3, St. Dev. =0.8348 , IQR=4.75-5) 

83. Plotting Software (Round Two Data: Mean=4.62 , Max=6, Min=3, St. Dev. =0.7679 , 
IQR=4-5) 

84. Presentation software (Round Two Data: Mean=4.85 , Max=6, Min=3, St. Dev. =0.8987 , 
IQR=4-5) 

85. Ability to abstract (Round Two Data: Mean=5.23 , Max=6, Min=4, St. Dev. =0.8321 , 
IQR=5-6) 

86. Ability to synthesize (Round Two Data: Mean=5.69 , Max=6, Min=5, St. Dev. =0.4804 , 
IQR=5-6) 

87. Analogical reasoning (Round Two Data: Mean=5.08 , Max=6, Min=4, St. Dev. =0.6686 , 
IQR=5-5.25) 
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88. Historical perspective (Round Two Data: Mean=4.46 , Max=6, Min=4, St. Dev. =0.8771 , 
IQR=4-5) 

89. Analysis-based design (Round Two Data: Mean=4.23 , Max=6, Min=3, St. Dev. =0.9268 , 
IQR=4) 

90. Basic mechanical mechanisms (Round Two Data: Mean=4.23 , Max=5, Min=3, St. 
Dev.=0.7250 , IQR=4-5) 

91. Failure mode and effects analysis (Round Two Data: Mean=3.85 , Max=6, Min=2, St. 
Dev.=0.9871 , IQR=3-4) 

92. Product Dissection (Round Two Data: Mean=5.23 , Max=6, Min=4, St. Dev. =0.5991 , 
IQR=5-6) 

93. Can you list additional skills, techniques, or engineering tools related to engineering 
design that secondary (high school) students should use to understand, manage, and solve 
technological problems? 

93a. Reverse engineering (new item, no Round Two data) 

93b. Finishing job to the last detail (new item, no Round Two data) 

93c. Recognizing team roles and personality types (new item, no Round Two data) 

If any of your answers in this section fell outside the IQR, please make comments so your 
colleagues can understand your position. Be sure to include the item number! 
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APPENDIX H 


Round 4 Email 
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You will be receiving an email from me shortly which will contain a link to the final 
survey associated with the Engineering Design in Technology Education study! To make this 
last survey as user friendly as possible, I only included items that had not met the criteria of 
mean stability and low IQR necessary for completion. Items included in this fourth round were 
unstable (mean shift of >15% between Rounds Two and Three), had a Round Three IQR>1, 
or had useful comments given in Round Three. I have attached your individual Round Three 
answers to this email for you to use when completing the Round Four survey. 

Thank you for your patience with this process and your kindness in giving your time to a 
stranger. I will email a final copy of the results to you in the next few weeks. 
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APPENDIX I 


Round 4 Survey Instrument 
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Round Four Survey 

The following seetion consists of responses to Question 1 : What aspects of the 
engineering design process best equip secondary students to understand, manage, 
and solve technological problems? For each item , please indicate your level of 
agreement with the following statement: This item is a critical and necessary 
component of an engineering design focused curriculum which is designed 
to equip secondary (high school) students to understand, manage, and solve 
technological problems. *The Round Three data is included with each question. 


2. Understand functional structures (Round Three Data: Mean= 4.00, Max=6, 
Min=l, St. Dev.= 1.5811, IQR=4-5) 

P 

Strongly Disagree (1) 

^ Disagree (2) 

Somewhat Disagree (3) 

P 

Somewhat Agree (4) 

^ Agree (5) 

Strongly Agree (6) 


6. Decision making methodologies (Round Three Data: Mean= 4.58, Max=6, 
Min=3, St. Dev.= 1.0836, IQR= 4-5.25) 

P 

Strongly Disagree (1) 

^ Disagree (2) 

Somewhat Disagree (3) 

P 

Somewhat Agree (4) 

^ Agree (5) 

Strongly Agree (6) 
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The additional items in the Round Four survey instrument are ineluded here without the Likert 
scale that was part of the original document as seen by the participants. This was done to make 
this document more readable. 

7. Written communication (Round three data: Mean=5.46, Max=6, Min.=4, Standard Dev. 
=0.6602, IQR= 5-6) Round Three Comments: We now that informal communication 
plays an under-recognized role in design team interactions. Although it is highly 
relevant, formal communication is overemphasized in design/engineering education- 
-maybe because it is easier to teach formal communication practices than informal 
communication practices. 

10. Negotiation (Round Three Data: Mean= 4.46, Max=6, Min=3, St. Dev.= 0.9674, IQR= 
4-5) Round Three Comments: There is a large body of evidence that suggests design, 
among many other things, IS negotiation. Negotiation requires mental flexibility, which is 
critical for being able to understand and reframe context 

13. MulticulturaFdiversity awareness (Round Three Data: Mean= 4.08, Max=6, Min=2, St. 
Dev.= 1.1152, IQR= 4) Round Three Comments: If one is to understand and relate to 
users from diverse backgrounds, and drive his/her design process with that understanding, 
one must have the highest awareness of multicultural issues. 

21. Engineering heuristics for/analysis-based design (Round Three Data: Mean= 3.75, 

Max=6, Min=l, St. Dev.= 1.3568, IQR= 3-4.25) 

26. Identiflcation of underlying scientiflc principles (Round Three Data: Mean= 4.92, Max=6, 
Min=4, St. Dev.= 0.8623, IQR= 4-6) 

27. Product optimization (Round Three Data: Mean= 3.00, Max=4, Min=l, St. Dev.= 0.8165, 
IQR= 3) 

39a. Costing, profit, and basic economic analysis (Round Three Data: Mean= 3.92, Max=5, 
Min=2, St. Dev.= 0.8623, IQR= 4) 

39b. Understanding the context of the technological problem and possible external influences 
(Round Three Data: Mean= 4.77, Max=6, Min=2, St. Dev.= 1.1658, IQR= 5) 

39c. Product architecture and modularity/interfaces 27. Product optimization (Round Three 
Data: Mean= 3.92, Max=6, Min=2, St. Dev.= 1.1875, IQR= 3-4) 

39d. Design principles to assist in generating innovative concepts (Round Three Data: Mean= 
4.38, Max=6, Min=3, St. Dev.= 0.9608, IQR= 4-5) 

39e. Design by analogy (Round Three Data: Mean= 4.00, Max=6, Min=3, St. Dev.= 0.8165, 
IQR= 4) 

39f. Understanding basic business motivations for engineering design, such as marketing or 
consumer research (New item, no round three data) 

39g. Understanding basic manufacturing processes (New item, no round three data) 

3 9h. House of Quality method (New item, no round three data) 

The next section consists of responses to Question 2: What mathematics concepts related to 
engineering design should secondary students use to understand, manage and solve technological 
problems? For each item below, please indicate your level of agreement with the following 
statement: This item is a critical and necessary component of an engineering design focused 
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curriculum designed to require secondary (high school) students to use various mathematical 
concepts to understand, manage, and solve technological problems. *The Round Three data is 
included with each question. 

46. Statistics (Round Three Data: Mean=4.23 , Max=6, Min=l, St. Dev.=1.5892, IQR=3-5) 

47. Calculus- Integration (Round Three Data: Mean=3.77 , Max=6, Min=l, St. Dev.=1.7394 , 
IQR=2-5) 

48. Calculus- Differentiation (Round Three Data: Mean=3.77 , Max=6, Min=l, St. 
Dev.=1.7394, IQR=2-5) 

49. Calculus- Differential Equations (Round Three Data: Mean=3.08 , Max=5, Min=l, St. 
Dev.=1.3205,IQR=2-4) 

50. Measurement theory (Round Three Data: Mean=3.46 , Max=5, Min=l St. Dev.=1.4500, 
IQR=2-4) 

57. Modeling/simulation/numerical analysis software (Round Three Data: Mean=3.92 , 
Max=6, Min=l, St. Dev.=1.5525, IQR=3-5) 

58a. Algebraic equations for determining gear ratios (Round Three Data: Mean= 2.75, Max=6, 
Min=l, St. Dev.= 1.4222, IQR= 1.75-3) 

58b. Conservation of momentum (Round Three Data: Mean= 4.25, Max=6, Min=l, St. 
Dev.=1.2154, IQR=(4-5) 

58c. Projectile motion (Round Three Data: Mean= 3.45, Max=6, Min=l, St. Dev.= 1.4397, 
IQR= 3-4) 

58d. Structural equilibrium (Round Three Data: Mean= 4.33, Max=6, Min=3, St. Dev.= 
0.9847, IQR= 4-5) 

58e. Basic stresses (Round Three Data: Mean= 4.25, Max=6, Min=2, St. Dev.= 1.1254, IQR= 
3.75-5) 

58f. Using geometry and trigonometry to change the scale of a component (New item, no 
round three data) 

58g. Formulas capable of expressing the performance of a system (New item, no round three 
data) 

The next section consists of responses to Question 3: What specific science principles related to 
engineering design should secondary students use to understand, manage and solve technological 
problems? For each item below, please indicate your level of agreement with the following 
statement: This item is a critical and necessary component of an engineering design focused 
eurriculum designed to require secondary (high school) students to use various scientific 
principles to understand, manage, and solve technological problems. *The Round Three data is 
included with each question. 

60. Effects of chemical formulation on manufacturing (Round Three Data: Mean=2.75 , 
Max=4, Min=l, St. Dev.=0.7538, IQR=2.75-3) 

61. Biological Evolution (Round Three Data: Mean=3.17 , Max=6, Min=l, St. Dev.=1.5859 , 
IQR=2.5-4) Comments from Round Three: For Q61, 1 put “1” since biological evolution 
has some scientific evidence to support it at the micro-evolution level but no evidence 

to support it at the macro-level. The concept of micro-evolution would be somewhat 
valuable to learn, but I don’t feel like the general concept of evolution would be valuable 
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62. Conservation of mass, energy, and momentum (Round Three Data: Mean= 4.92, Max=6, 
Min=2, St. Dev.= 1.2401, IQR= 4-6) 

69. Control theory (Round Three Data: Mean= 2.83, Max=5, Min=l, St. Dev.= 1.3371, IQR= 
2.5-3) Round Three Comments: Control theory is applicable to all engineering domains 
and is critical for understanding the behavior of all engineering systems. 

74. Strength of materials (Round Three Data: Mean=4.25 , Max=6, Min=2, St. Dev.=1.1382, 
IQR=3.75-5) 

75. Thermodynamics (Round Three Data: Mean=3.25 , Max=5, Min=2, St. Dev.=0.8660, 
IQR=3-4) 

76. Decision analysis (Round Three Data: Mean=3.33 , Max=5, Min=l, St. Dev.=1.1547, 
IQR=3-4) Round Three Comments: Decision analysis is applicable to design practice 
in all engineering domains, and facilitates the application of well-considered design 
processes. 

78. Learning theories (Round Three Data: Mean=3.58 , Max=6, Min=l, St. Dev.=1.6214 , 
IQR=3-4.5) Round Three Comments: Design is a learning-intensive activity. One cannot 
master designing without understanding his/her learning processes 
79a. Project management (Round Three Data: Mean= 4.17, Max=6, Min=2, St. Dev.= 1.2673, 
IQR= 3-5) 

79b. Thermal expansion/contraction (Round Three Data: Mean= 3.92, Max=5, Min=2, St. 
Dev.= 0.7930, IQR=4) 

79c. Question asking -inquiry (New item, no round three data) 

79d. Leadership principles (New item, no round three data) 

7 9e. Principles related to environmental consciousness (New item, no round three data) 

The last section consists of responses to Question 4: What specific skills, techniques, and 
engineering tools related to engineering design should secondary students use to understand, 
manage and solve technological problems? For each item below, please indicate your level of 
agreement with the following statement: This item is a critical and necessary component of an 
engineering design focused curriculum designed to require secondary (high school) students 
to use various skills, techniques, and engineering tools to understand, manage, and solve 
technological problems. *The Round Three data is included with each question. 

80. Computer aided design software (Round Three Data: Mean=4.33 , Max=6, Min=2, St. 
Dev.=1.2309, IQR=3.75-5) 

81. Computer searching (Round Three Data: Mean=4.92 , Max=6, Min=4, St. Dev.=.7930, 
IQR=4-5.25) 

89. Analysis-based design (Round Three Data: Mean=4.25 , Max=6, Min=l, St. 

Dev.=L4222, IQR=4-5.25) 

91. Failure mode and effects analysis (Round Three Data: Mean=3.00 , Max=5, Min=l, St. 
Dev.=1.2060, IQR=2.75-4) 

93a. Reverse engineering (Round Three Data: Mean= 4.17, Max=6, Min=l, St. Dev.= 1.4668, 
IQR= 4-5) 

93b. Finishing job to the last detail (Round Three Data: Mean= 3.92, Max=6, Min=l, St. 

Dev.= 1.4434, IQR=3-5) 
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93c. Recognizing team roles and personality types (Round Three Data: Mean= 4.58, Max=6, 
Min=4, St. Dev.= 0.7930, IQR=4-5) 

93d. Engineering intuition (New item, no round three data) 

As planned, this final survey concludes the data collection portion of the Engineering 
Design in Technology Education study! I am most appreciative of your time and support 
- this study simply could not have been completed without your kindness. Thank you very 
much, Cameron Smith 
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